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Abstract. We consider the three dimensional gravitational Vlasov Poisson sys- 
tem which describes the mechanical state of a stellar system subject to its own 
gravity. A well-known conjecture in astrophysics is that the steady state solu- 
tions which are nonincreasing functions of their microscopic energy are nonlin- 
early stable by the flow. This was proved at the linear level by several authors 
based on the pioneering work by Antonov in 1961. Since then, standard varia- 
tional techniques based on concentration compactness methods as introduced by 
P.-L. Lions in 1983 have led to the nonlinear stability of subclasses of stationary 
solutions of ground state type. 

In this paper, inspired by pioneering works from the physics litterature |41) . 
[54] , [5], we use the monotonicity of the Hamiltonian under generalized symmet- 
ric rearrangement transformations to prove that non increasing steady solutions 
are local minimizer of the Hamiltonian under equimeasurable constraints, and 
extract compactness from suitable minimizing sequences. This implies the non- 
linear stability of nonincreasing anisotropic steady states under radially symmet- 
ric perturbations. 



1. Introduction and main results 

1.1. Setting of the problem. We consider the three dimensional gravitational 
Vlasov-Poisson system 



^1.1) 



dtf + v- V^f - Vcpf ■ Vyf = 0, {t, X, v) £R+x x 
f{t = 0,x,v) = fo{x,v)>0, 

where, throughout this paper, 

Pfi^) = I f{x,v)dv and cj)f{x) = -—^*pf (1.2) 

JR3 47r|X| 

are the density and the gravitational Poisson field associated to /. This nonlinear 
transport equation is a well known model in astrophysics for the description of the 
mechanical state of a stellar system subject to its own gravity and the dynamics of 
galaxies, see for instance |10| I15j. 



Unique global classical solutions for initial data fo ^ Cl, fo > 0, where denotes 
the space of compactly supported and continuously differentiable functions, have 
been shown to exist in |401 Wl\ 09] and to propagate the corresponding regularity. 



Two fundamental properties of the nonlinear transport flow (1.1) are then first the 
preservation of the total Hamiltonian 

nf{t)) = \l \v\^f{t,x,v)dxdv-l [ \V<Pf{t,x)\^dx = nifm, (1.3) 



2 



M. LEMOU, F. MEHATS, AND P. RAPHAEL 



and second the preservation of all the so-called Casimir functions: VG G C^i[0, +00), M+) 
such that G(0) = 0, 

/ G{fit,x,v))dxdv = [ G{fo{x,v))dxdv . (1.4) 

This last property induces a continuum of conservation laws and is the major dif- 
ference between this kind of problem and other nonlinear dispersive problems like 
nonlinear wave or Schrodinger equations. 

1.2. Nonlinear stability of steady state solutions. A classical problem which 
has attracted a considerable amount of work both in the astrophysical HI [23 
UU H2I [5JI [2] and mathematical communities, is the question of the nonlinear sta- 
bility of stationary states. If we restrict our study to radially symmetric stationary 
states -that is a priori depending on |f |, x • v) only-, Jean's theorem [8] ensures 
that they can be described as functions of their own microscopic energy and their 
angular momentum: 

e{x,v) = — + (I)q{x), £{x,v) = \x X vl"^, (1.5) 



Qix,v) = F{e{x,v),£{x,v)). (1.6) 

The existence of such steady states has been discussed in [8] for a large class of 
smooth functions F. A well-known conjecture in astrophysics, [10}, is now that 
among these stationary solutions, those who are nonincreasing functions of their 
microscopic energy e are nonlinearly stable by the Vlasov Poisson flow, explicitly: 

Conjecture: Non increasing anisotropic galaxies F = F{e,i) with ^ < on the 



support of Q are stable by spherically symmetric perturbations for the flow (1.1). 
Non increasing isotropic spherical galaxies F = F(e) with ^ < on the support 



of Q are orbitally stable against general perturbations for the flow (1.1). 



Remarkably enough, this conjecture has been proved at the linear level by Dore- 
mus, Baumann and Feix (see also |17| |25| [52] for related works), following the 
pioneering work by Antonov in the 60's [3l [1]. These results are based on some co- 
ercivity properties of the linearized Hamiltonian under constraints formally arising 



from the linearization of the Casimir conservation laws (1.4), see Lynden-Bell [41| . 

At the nonlinear level, the general problem is open. However, the nonlinear 
stability of a large class of stationary solutions of so-called ground state type in- 
cluding the polytropic states has been obtained using variational methods in |55| . 
[m [201 [211 [221 [13], completed by [50]. In [281 [291 [30] , see also [IE], we observed 
that a direct application of Lion's concentration compactness techniques |381 [39], 
implies that or a large class of convex functions j, the two parameters -according 



to the scaling symmetry of (1.1)- minimization problem 

/(Mi,M,)= inf nif), Mi,Mj>0 (1.7) 



is attained up to symmetries on a steady state solution to (1.1) of the form (1.6) 



and all minimizing sequences to (1.7) are relatively compact up to a translation 
shift in the natural energy space 

£ = {f>0 with \f\£ = l/l^i + \f\L^ + \\v\^f\Li < +00}. 

The so-called Cazenave, Lions |11J theory of orbital stability then immediately im- 
plies the orbital stability of the corresponding ground state steady solution, |29| . 
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In fact, this last step requires the knowledge of the uniqueness of the minimizer to 



(1.7) which is a delicate open problem in general, see [50] . but this difficulty was 



overcome in E 



Other non variational approaches based on linearization techniques have also been 
explored in |53| 123) . Recently, Guo and Lin [19] proved the radial stability of the 
so called King model F{e) = (exp(eo — e) — 1)+ which is not in the class of ground 



states as obtained in the framework of (1.7). Adapting a robust approach developped 



by Lin and Strauss in their study of the Vlasov Maxwell system, \35\ \36[ I37| . the 
authors use the infinity of conservation laws provided by the nonlinear transport to 
construct a sufficient large approximation of the kernel of the linearized operator 
close to the steady state. This allows them to recover a coercivity statement of 
the linearized energy using Antonov's coercitivity property which after linearization 
and control of higher order terms for the King model yields the claimed stability in 
the radial class. 

1.3. Additional conserved quantities in the radial setting. Our main purpose 
in this paper is to describe a generalized variational approach for the nonlinear 
stability of steady states which fully takes into account the nonlinear transport 
structure of the problem, and in particular the continuum of constraints at hand 



from (1.4). 

First recall that in general, the full set of invariant quantities conserved by the 
nonlinear transport flow ( |l.ip depends on the initial data and its possible symme- 
tries. From now and for the rest of this paper, we shall restrict our attention to 
spherically symmetric solutions f{x,v) = f{\x\, \v\,x-v) where we will systematically 
abuse notations and identify / with its image through various diffeomorphisms. We 
then let £rad be the space of spherically symmetric distribution functions of finite 
energy 

£rad = {/ S iS, / spherically symmetric}, (1-8) 

and recall that if / is spherically symmetric, then Pf{x) = pf{\x\) and (j)f{x) = 
(j)f{\x\). This implies in particular from a direct computation that the momentum 

l=\x x -uP 



is conserved by the characteristic flow associated to (1.1), and hence a larger class 



of Casimir conservation laws (1.4) holds: 

G{f{t,x,v),\xxv\'^)dxdv= I G{fQ{x,v),\x x v\'^)dxdv (1.9) 



for all G G Ci([0,+oo) x [0,+oo),M+) with G{<d,l) = 0, > 0. 



Let us reformulate (1.9) in terms of equimeasurability properties of / and /( 



0- 



Performing the change of variables 

r = |x|, i(; = |f|, X ■ V = \x\\v\ cos 6 , r,w > 0, ^G]0,7r[, 
the Lebesgue measure is mapped onto: 

foo z'+oo fir 

f{x,v)dxdv = 8TT'^ / / / f {r,w, COS 9) r'^w'^ sin 9 drdwdO. 
Jr=o Jw=o Je=o 

We then perform the second change of variables 

r = r, ti = w sign (cos £ = r'^w'^ sin^ 9 
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and get from Fubini: 

di (1.10) 



r>+00 

/(x, v)dxdv 



1=0 



f{r,u,£)diye 

(r,«)gf2£ 



with 

fi^ = {(r,u) G M+ X M with r'^u'^ > £} (1.11) 

and 

di^l, = 47r2l^2„2>^ (r^u^ _ i)-'^/'^r\u\drdu. (1.12) 

We then define the distribution function of / at given kinetic momentum i: \/i > 0, 
Vs > 0, 

fif{s,£) = ue{{r,u) GQe, f{r,u,£)>s}, (1.13) 

or equivalently 



OO /" + 00 



^lfis,£) =4tt^ / l/(^,„,£)>,(rV -£) ^/2^|w|1^2„2>^drd'u. (1.14) 

J r=0 Ju=—oo 

We now define the set of distribution functions which are equimeasurable to / at 
given £ by: 

Eq(/) = {g > spherically symmetric, Vs > 0, fif{s,£) = fig{s,£) a.e. £}. 

(1.15) 

We then have from standard arguments: 

Lemma 1.1 (Characterization of Eq(/)). Let f £ L^r\L°° , nonnegative and spher- 
ically symmetric, then the following are equivalent: 

(i) g G Eq(/); 

(ii) \/G{h,£) > 0, with G{0,£) = 0, there holds: 



G{f{x,v),\xxv\ )dxdv = / G{g{x,v),\x x v\ )dxdv. 
Lemma [1.11 allows us to reformulate the conservation laws of the full Casimir class 



(1.9) in the radial setting as follows: 

Vt>0, /(t)GEq(/o). (1.16) 

1.4. Assumption (A) on the steady state. Before stating the results, let us fix 
our assumptions on the steady state Q. 

(i) (5 is a continuous, nonnegative, non zero, compactly supported steady state 
solution of the Vlasov-Poisson system (1.1). 

(ii) There exists a continuous function F : M x ]R+ — t- M-|- such that 

V(x,?;)gM6, Q{x,v) = f(^^ + cpq{x),\xxv\'^^. (1.17) 

(iii) There exists eo < such that: 

O = {(e, £) G M X M+ : F(e, £) > 0} c] - oo, eo[xM+, 
dF 

F is on O, with ^ < 0. 

oe 

Remark 1.2. Note that ^ may be infinite at the boundary of O, as is the case 
for polytropic ground states F{e,£) = (eo — e)'^£'^, for some < q < 1 and k>0. 
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Below we list a number of physically relevant models for which our non linear 
stability result applies. All these examples are extracted from [10] to which we refer 
for a detailed physical description of various gravitational models. 

Examples. 

- Polytropes and double-power models: The polytropes correspond to the 
following form of F: 

F{e, e) = (eo - e)^r, 0<q< 7/2, k > 0, 

where eo < is a constant threshold energy. A generalization of these 
polytropes is provided by the so-called double-power model jlOj: 

F{e,£)= «.,(eo-e)^^^ 

where aij are nonnegative constants. 

- Michie-King models: 

F{e,i) = exp(-£/2r2) (exp(eo - e) - 1)+ , 

where eo < and the constant > is the anisotropy radius [10]. When 
Ta goes to infinity, this model reduces to the King model. 

- Osipkov-Merritt models: 



F{e,£) = G{eo-e+—^ 



where eo < 0, > are constants, and G is a nonincreasing function 
such that G{t) =0 for all t < 0. 



1.5. Statement of the results. From (1.16), a natural generalization of (1.7) in 



the radial setting is to minimize the Hamiltonian under constraints of given equimea- 
surability. This is a very natural strategy to prove stability in a nonlinear transport 
setting which goes back in fluid mechanics to the celebrated works of Arnold, see 
e.g [5], [6], [7], Marchioro and Pulvirenti |43) . |45| , Wolansky and Ghil |56j . and 
references therein, and is also very much present in the physics litterature, see in 
particular Lynden-bell |3T], Gardner [16], Wiechen, Ziegler, Schindler [54], Aly |2] 
and references therein. The mathematical implementation of the corresponding 
variational problem is however confronted to the description of bounded sequences 
in Eq(/o) and a possible lack of compactness in general, see for example Alvino, 
Trombetti and Lions [1] for an introduction to this kind of problem. 

Our first result is the characterization of non increasing states as local minimizers 
of the Hamiltonian in S^ad under a constraint of equimeasurability: 

Theorem 1.3 (Local variational characterization of Q). There exists a constant 
Co > such that the following holds. For all R > 0, there exists 6q{R) > such 
that, for all f G Srad H Eq((5) satisfying 

\f-Q\£<R, \V4>f-V4>Q\v^<So{R), (1.18) 

we have 

n{f)-'H{Q)>Co\V(Pf-V^Q\l2. (1.19) 
// in addition 1-L{f) = 1-L{Q), then f = Q. 
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Theorem 1.3 was first obtained by Guo, Rein |23) for a perturbation / near 



in the specific case of the isotropic King model, and for isotropic relativistic models 
F[e,l) = F{e) with locally bounded derivative F'[e) in [24j, and this excludes any 
singularity at the boundary -as many polytropic models would have-. 



Let us stress onto the fact that Theorem 1.3 by itself alone is too weak to yield a 



stability statement including the full set of radial pertubations. Hence the impor- 



tance of Theorem 1.3 relies in fact mostly on its proof. Indeed, a new important 
feature of our analysis is to use a monotonicity property of the Hamiltonian under 
a generalized Schwarz symmetrization which is not the standard radial rearrange- 
ment but a rearrangement with respect to a given microscopic energy — |-</>(x), at 



. . 2 

|2 



fixed angular momentum \x x v\ , see Proposition 2.8 for a precise definition and 



Proposition 3.1 for the monotonicity statement. This monotonicity is very much a 
consequence of the "bathtub" principle for symmetric rearrangements, see Lieb and 
Loss ^33j, and was already observed in the physics litterature, see Gardner |16j . Aly 
|2]. It produces a reduced functional which depends on the Poisson field (f)f 

only and not the full distribution function. The outcome is a lower bound 

nf)-nQ)>j{<pf)-j{4>Q)- (1.20) 

Interestingly enough, the reduced functional J was first introduced on physical 
ground as a generalized potential energy in the pioneering works by Lynden-Bell 
|41] . see also Wiechen, Ziegler, Schindler |5"i] . It now turns out from explicit com- 
putation that the critical points of J' are the Poisson field of steady states, and that 
the Hessian of J7 near the Poisson field of a nondecreasing steady state can be di- 
rectly connected to the Hartree-Fock exchange operator |41j . which is coercive from 
Antonov's stability criterion, see section |4| and hence (pQ itself is a local minimizer 
of J. 



The important outcome of the structure (1.20) is that by reducing the problem 
to a problem on the Poisson field only, we are able to extract compactness in the 
radial setting from any minimizing sequence whose Hamiltonian converges to Q 
without the assumption of equimeasurability, thanks to the smoothing and compact- 
ness provided by the radial Poisson equation. This allows us to prove the following 
compactness result which is the heart of our analysis. 

Given / E £rad, we consider the family of its Schwarz symmetrizations f* {■,£), 



> 0, as defined in Proposition 2.6 We then claim: 



Theorem 1.4 (Compactness of local minimizing sequences). There exists 6 > 
such that the following holds. Let fn be a sequence of functions of S^adj bounded in 
L°°, such that 

|V<^/„-V</.qU2 <<5, (1.21) 

and 

lim supTiifn) <'H{Q), f*^Q*tnL^{R+xR+) as n ^ +oo (1.22) 

n— >+oo 

then 

fn^QtnL\R^), \v\'^fn^\v\^Q tn L\R^). (1.23) 



Theorem |1.4| is the key to the radial Cazenave-Lions' theory of orbital stability 
IT] and implies that any compactly supported non increasing steady state Q as 



^and not only <j}f near (pg which is an issue for the proof of Theorem 1.4 
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defined by (1.17), is nonlinearly stable under the action of the Vlasov-Poisson flow 
with respect to spherical perturbations. We thus obtain the main result of this 
paper: 



Theorem 1.5 (Nonhnear stability of Q under the nonlinear flow (1.1)). For all M 

large enough and for all e > 0, there exists rj > such that the following holds true. 
Let /o G Srad n Cl with 

\fo-Q\L^<V, |/o|l-<M, nifo) <'H{Q) + v, (1.24) 

then the corresponding global strong solution f{t) to (1.1) satisfies: \/t > 0, 

\fit)-Q\L^ + \\v\^{f{t)-Q)\Li<e, \fit)\L^<M. (1.25) 



Comments on Theorem 11.51 

1. Linear versus nonlinear stability. A natural strategy to pass from linear to 
nonlinear stability is to try to linearize the problem and estimate higher order terms 
as perturbations. This turns out to be quite delicate in general and the control of 
higher order terms may be challenging, see |53) . |19] for a treatment of the King 
model, |32] for the polytropic case. Our analysis avoids this classical difficulty using 
two facts. We first derive a global monotonicity property which is fundamentally 
a nonlinear property and does not rely on any linearization procedure. Proposition 



3.1 and which reduces the problem to understanding a simpler functional on the 
Poisson field (p only. For this functional, we do apply a linearization procedure that 
is a Taylor expansion near (j)Q, but we avoid the computation of higher order terms 



thanks to compactness properties of the Hessian, see (4.45), (4.61) 



2. Comparison with previous nonlinear stability results. In view of the nonlinear 



stability result obtained for ground state type minimizers of (1.7) which are not 



restricted to the radial class, one may ask whether a generic steady solution of the 



form (1.17) can in fact be obtained as a ground state for (1.7). This is a nontriv- 
ial issue which is connected to the notion of equivalence of ensemble in statistical 
physics. In a forthcoming work [31 J and following pioneering ideas from Lieb and 
Yau |33] , we will exhibit a large class of monotonic functions F for which the equiv- 
alence of ensemble actually holds. There are however of course many well known 
examples where this equivalence of ensembles fails. Note also that physical investi- 
gations around these minimization problems can be found in [12] and the references 
therein. 

3. Comparison with 2D incompressible Euler. The conservation of equimeasurabil- 
ity properties by the nonlinear transport flow has also been used in the literature 
to prove the stability of steady states for the 2D incompressible Euler flow, see 
for example Marchioro, Pulvirenti [45] and references therein. For a discussion on 
variational problems with equimeasurability constraints in fluid dynamics, one can 
also refer to Serre [51j. Our result generalizes this approach to the Vlasov-Poisson 
system which is however more delicate due to the non trivial structure of both the 
Hamiltonian and the steady states solutions. 



The conjecture of stability of nonincreasing radially symmetric steady states is 
hence proved for radial perturbations. Note that the result is expected to be opti- 
mal for anisotropic galaxies with a non trivial dependence on i as some numerical 
simulations suggest the possible instability of anisotropic models against general 
perturbations, see |10j . One important open problem after this work is certainly 
the general setting of nonradial perturbations for spherical models. 
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1.6. Strategy of the proof. Let us give a brief insight into the proof of the varia- 



tional characterization of Q given by Theorem 1.3 and the lower bound (1.20) which 



are key features of our analysis. It follows in three main steps. 
Step 1. Rearrangement with respect to a given Poisson field. 

Let a Poisson field (j) and a radially symmetric distribution function / G £rad^ we 
aim at defining the Schwarz symmetrization of / with respect to the microscopic 
energy e = — h (/>(x) at each given kinetic momentum i. In other words, given 



{x, v) = Gy^ + (j){x),ej which 
is a nonincreasing function of e and which is equimeasurable to / in the sense of 



i = \x X v\'^ > 0, we are looking for a function /* 



(1.13), (1.15) i.e.: 



Vt > 0, fif{t,£) = ijf,4,{t,£) a.e £ > 0. 



As a simple change of variables formula similar to (1.10) reveals, the choice of f*'' 
is essentially unique and given by: 

,|2 



r'i'{x,v) = r 



+ 4>{x), \x X v\ \ ,\x X V 



1 



J^+</>(x)<0 



:i.26) 



where /* is the standard Schwarz symmetrization of / at given £ -see Proposition 2.7 
for a precise statement- and is the Jacobian of the change of variables, explicitly: 

£ V^^ 



r+co 

e,£) = 87r2\/2 / 



Note that the steady state Q being by assumption a nonincreasing function of its 
microscopic energy, it is automatically a fixed point for this transformation -see 
Corollary [2:9]-: 

Q = Q*'^Q. (1.28) 
Step 2. Monotonicity of the Hamiltonian under the f*'f'f rearrangement. 

The key property which can be found in the physics litterature, see in particular 



Aly |2], is now the monotonicity of the Hamiltonian (1.3) under the generalized 
rearrangement (1.26), see Proposition |3.1| 

"^feSrad, n{f)>n{r^n- (1-29) 

Pick then / as in the hypothesis of Theorem |1.3| so that /* = Q*, then a slightly 



more careful analysis of the monotonicity formula (1.29) implies a lower bound of 



the Hamiltonian by a functional which depends on the Poisson field (pf only: 

nf)-n{Q)>J{^f)-J{<PQ) (1.30) 

with 



+ 



1 



This dependence in 4> only which displays nice compactness properties in the radial 
setting is the key to the proof of the convergence of minimizing sequences. The- 
orem 1.4 Another important feature in the proof of Theorem 1.4 will be to not 



only use the monotonicity ( 1.29 ), but to observe that some norm is controlled by 
^(/) - 'H{f*'t'f), see section |3.3| 



Step 3. Coercivity of the Hessian of J at cpq. 
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From (1.30), the lower bound ( |1.19 ) now follows from the lower bomid: 

J{<Pf)-J{(t>Q)>C\V^f-V<f>Q\l, (1.31) 

in the vicinity of (pq. This local coercivity lower bound relies on an explicit com- 
putation of the Taylor expansion of J at (pg, Proposition 

is a critical point of J, whil 



4.3 



equation (1.28) implies that 



The steady state 
;he Hessian at cpq is 

intimately related to the Lynden-Bell Hartree-Fock exchange operator |41j , which 



e t 



coercivity was essentially proved 40 years ago by Antonov, |3], |3], see Proposition 



4.1 using in particular the fact that in radial symmetry, the kernel of the linearized 
transport operator close to Q is explicit. Note that the rigorous derivation of the 
first two derivatives of J at c pq req uires a detailed study of the regularity properties 
of the Jacobian given by (1.27) which a priori displays a -^A regularity only. 



This paper is organized as follows. In section |2] we introduce the Schwarz sym- 



metrization with respect to the microscopic energy 



+ 



at fixed kinetic 



momentum |x x t;| , and prove some natural continuity property of the correspond- 



ing object Z*"^, Proposition 



2.8 



and of the Jacobian function a^, Lemmas 



2.3 



2.5 In section [3j we prove the key monotonicity Proposition 3.1 which reduces 



2.4 



the 



analysis to coercivity properties of the functional J^{4>) near (j)Q, Proposition 3.2 



We then first conclude the proofs of Theorems |1.3[ |1.4[ |1.5| assuming Proposition 
3.2 which is eventually proved in section |4j 
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2. Symmetric rearrangement with respect to a given microscopic 

energy 

Our aim in this section is to introduce the symmetric rearrangement f*'^ of a 
distribution function / G S^ad with respect to a given microscopic energy 



+ 



4>{x). This notion generalizes the standard Schwarz symmetrization and is the well 



fitted object for the study of the minimization problem (1.19). This symmetrization 



involves the use the Jacobian function a^f, given by (1.27). We start with proving 



some continuity and differenti abil i ty proper ties of this functional which will be used 



all along the paper. Lemma 
properties. Proposition |2.8[ 



2.3 



2.4 



2.5 



and then define J*"^ and give its first 



Our 



2.1. Definition and differentiability in e of the Jacobian function 

aim in this subsection is to study the Jacobian a^(e, given by (1.27), which appears 
in the definition of the generalized Schwarz symmetrization (2.63). The class of 
Poisson potentials (j) which is well fitted for the analysis is 

^rad = {i?5> : — )■ M such that there exists / G E^ad with cj) = (pf^ (2-1) 

Let us start with some properties of the so called effective potential appearing in 



the definition (1.27) which are elementary but crucial to obtain uniform bounds on 



and its various derivatives. 
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Figure 1. Profile of tlie effective potential ip^/{r) 

Lemma 2.1 (Structure of the effective potential for (p G ^rad)- Let (p = (pf G ^radi 
be non zero. For i > 0, consider the effective potential 



(i) Structure of ip^^^: ipfj,^£ G C^(M'^\{0}) and 

r>0 



r > 0. 



(2.2) 



(2.3) 



is attained at a unique ro((p,i). ip^^£ is strictly decreasing on (0, ro((/>, ^)) and strictly 
increasing on (rg ((/>,£), +oo) with 

lim V'0/(r) = +00, lim ^^ £{r)=0. (2.4) 

r— s>0 1 — 5>+oo 

Moreover, the function £ i— )• e^/ is continuous on M!j_, with the uniform bound: 

yi > 0, max U{0), < e^,e < 0. (2.5) 



(a) Level sets ofip^p^g: for e^/ < e <0, let 

n {(f>, e, e) = inf {r>0 St. e- ip<t>,e{r) > 0} , (2.6) 

r2{(j), e, t) = sup {r > St. e - ^ci>/{r) > 0} . (2.7) 
Then ri(<l),e,i),r2{(j),e,£) are functions of e with uniform bounds: Ve^^^ < e < 0; 

e 



0<-rj—<ri{<p,e,i)<r2icl),e,l) < 



(2.8) 



(Hi) Concavity lower bound: there holds the uniform concavity lower bound Ve G 
{e^,i,0), Vr G [ri(e, (/>,£), r2(e, £)], 



2r^ rir2 



(r - ri{(p,e,£)){r2i(l),e,£) - r). 



(2.9) 



On Figure [T] we summarize the properties of ipi described above. 



Remark 2.2. In the sequel and when there is no ambiguity, we will avoid the 
{(p,e,£) dependence and note rQ,ri,r2- 

Proof. The proof is elementary but relies on a crucial way on the positivity of A<j)f. 
Let us recall the standard interpolation estimate for f G £: 



\yh\h<c\\v\'f\'/M/.'\f\]!^ 



(2.10) 
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Let 



G ^radi then by interpolation and Sobolev embedding, p/ G 



and thus St £ W, 



2'5/3/Tn,3\ 



loc 



C C"(M^) and 4>f G Ci(M-^\{0}) by elhptic regularity and 



the radial assumption, from which i/j^^^ G C^(M^\{0}). 
We now integrate the radial Poisson equation and get: 



r^(f,'j^{r) = 47ry s^pf{s)ds > 0, 



lim r(pf{r) 

r— >4-oo 



(2.11) 



Note that the second identity is obtained by integrating the first one as follows: 

pr f+oo 

r(l)f{r) = —An / pf{s)ds — Aixr i spf{s)ds. (2.12) 

Jo Jr 

We deduce that (p = (pj is continuous, nondecreasing and nonpositive on [0, +oo[ 
with 

and there exists f > 0, such that 



(r) > max ( (/>(0), 



Vr > 0, 



(2.13) 



r < 



2r 



V r > f. 



(2.14) 



Thus ( |2.13[ ), ( |2.14[ ) imply (|2^. From ( |2.14| , e,^,^ given by ([2^ satisfies 



60/ < inf 

r>f 



2r 2r2 



<0, 



since by assumption / 7^ 0, and hence e^/ is attained at some ro = ro(e, I). Thus 
from dfel): 



e0,£ = 0(ro) + TTT > max ( (t>{0), 



+ — I > max (0(0),-^ 



2rg 



> 0, 



and (2.5) is proved. 



Observe now from (2.11) again that: 

i. i 
'^'<I>A'^) = ^'(r) - and (r^ip'^/r))' = r'^ pf + > 0, 



and hence from i/^^(ro(e, = 0: 



(2.15) 



(2.16) 



which yields the uniqueness of the minimum tq > and the claimed monotonicity 
properties of ■0</>/- 



Together with (2.4), we conclude from (2.16) that ri,r2 given by (2.6), (2.7) are 
well defined for e^^i < e < 0, and are functions of e from the implicit function 
theorem. To prove the uniform bound (2.8), we observe from (2.13): 



r > 0; st. e 



> 



and hence using from (2.5) that |/|?i + 2e£ > for e > es/'- 



r > 0: st. e + 



2r2 



> 0^ C 



l/lLi + A/l/lfi+2e^ l/lLi-A/l/lfi+2e£ 
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We then use the definitions (2.6) and (2.7) to get 

< 



l/lLi+A/l/lii+2e£ 



<ri(0,e,£) <r2((/>,e,^) < 



l/lLi-A/l/lii+2e£ 



which implies (2.8). 

Let us now prove the continuity of the function 1 1— )• e^ g on M^. Let < < £2 
be fixed. From the definitions (2.2) and (2.3), for all i S [^1,^2] we have 



thus, applying (2.8) with e = ^e^^^j gives 
tti < ^"0(0)^) < with a\ - 



4 _ _ 2|/|^i 



2|/li 



> 0, 02 



I 60/2 I 



> 0. 



Hence, (r, £) 1— )• ^^/(r) being continuous, the function 

t G [4,^2] e,^,^ = min ^^^ir) 

re[a,02] 

is continuous. 

It remains to prove the concavity bound (2.9). Let 

w{r) = e - i'd./ir) - (r - ri)(r2 - r), 

zr^ rir2 

then 



{rw{r))" = ^ i -(r'^^j'^/r))' + 4 ) = -rpf{r) < 



where we used (2.15). Hence the function r 1— t- rw{r) is concave. Since it vanishes 
at ri and r2, we conclude that w{r) > for all r G [ri,r2] and (2.9) is proved. 

□ 



This concludes the proof of Lemma 2.1 



Let us now define the Jacobian function 0^(6,^) and examine its differentiability 
properties in e : 

Lemma 2.3 (Definition and differentiability properties in e of the Jacobian a^). 
For (p = (pf £ ^rad '^on zero and i > 0, we define: 



f i/^ I (r, u) G (M+)2 : + 0(r) < e| for e < and £ > 0, 
I +00 for e > 0, and £ > 0, 



where is the measure given by (1.12), equivalently: \/£ > 0, Ve < 0, 
a^{e, £) = 87r2^/2 ! \e - il>4,,i{r)f/^ dr. 



Then: 

(i) Behavior of a^: a^{e,£) = for e < e^^^ and: 
\/£ > 0, 

(a) Uniform bounds on a^: let 



> 0, a^{e^/,£) = 0, lim aJe,£) = +00. 

e— s>0^ 



< m0 := inf (r + l)|</'(r)| < +00, 

then there holds the bounds: 

Ve<0, a^{e,£) <167r^\e\-^^^\ f\Li, 

and 



(2.17) 
(2.18) 

(2.19) 

(2.20) 

(2.21) 
(2.22) 



13 



(iii) Differentiability in e: the map e — >• a^{e, t) is a -diffeomorphims from (e^^^, 0) 
to (0, +oo) with: 



Ve G]e^,,,0[, ^(e,^) = ^^^^ " ^<t>Ar))~^'^ dr > 0. 



(2.23) 



Abusing notations, we shall denote in the sequel a^^{-,i) : (0, +oo) — )• (cip/jO) its 
inverse function. 

Proof. Step 1. Bounds on a^. 

First compute from the definitions (2.17) and ( |1.12 ): Ve < 0, > : 



e, I) = Svr^ 



r>0 Ju>0 



lj,2u2>^ (r^u^ — i) ^^"^rudrdu 

\/2(e-0(r)) 



(r^u'^ — £) ^^^udu ) rdr 



87r2\/2 



^'2 



this is (2.18) or, equivalently, (1.27). Then a^{e,£) = for e < e^^^ and O0(e,^) > 
on {Cfij^i, 0) from Lemma 



2.1 



We now estimate from above for e < using (|2.13|) and (|2.8|) as follows 



ri ((p,e,i: 



2r2 J "^"^ 



Jr 



I/I 



1/2 



2| ^1 .|„|-l/2 



< 167r^l/lLi|e 



and (2.21) is proved. To estimate a^{e,£) from below, first observe that (2.20) 

\ 1/2 



follows from (2.11). We then write: 



+ 00 



e + 



r + l 



2r2 J 



»+oo 



> 87r2\/2 / 
and observe that for r > 1 + £/m^ , we have 



e + 



dr 



r + 



1 2r2 / 



> 



> 



r + l 2r2-r + l 2(r2 - 1) " r + 1 V 2m<^(r - 1) y " 2(r + 1) ' 



dr 



> 



Thus: 



+ 00 



e + 



2(r + l) 



1/2 



/ Ipl \ 1/2 /■+00 

> 87r2V2^ / (m^-2\e\{r + l))^fdr 

\m^J Ji+e/m^ 



> 87r2 — ler^/^m,^ ( 1 - 2|e 



\ 3/2 

2m0 + £ \ 
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This means that for |e| < 4(2^^+^ , a<),iej) > ^\e\~^^^rn^, and (2.19) and (2.22) 
are proved. The continuity and the monotonicity of the apphcation e 1— t- a(^(e, is 
a consequence of (2.8) and of the dominated convergence theorem, since 

e - '^(0 - 2^ j < (-0(0))'/', for all r e]e^,,, 0[. 



Step 2. Differentiability of < 



We are now in position to prove the differentiability of the function e 
which follows from the version of Lebesgue's derivation theorem given by Lemma 
|A.1[ Let us fix £ > and write 

f+00 



g{e, r)dr 



with 

g[e,r 



(e, r) = 87r2\/2 (e - ilj^^fXr))'^^'^ lri(0,e/)<r<r2(0,e,£) = 87r2\/2 (e - ^0,^?^))+ ^ , 



and with ip^^i given by (2.2). Let cq G]e(^^£,0[ and / =]eo — e, eo + e[, with e small 
enough such that / c]e(^_£,0[. Let us check the assumptions of Lemma A.l By 
(2.13), we have 

1/2 



0<g{e,r)<C 



I/I 



c 



V 2|eQ+£| 



I/Ili1 



where we used (2.8). Hence, by standard dominated convergence, g G C^{I, -L^(M+)). 
Moreover, from the regularity of the boundary ri,r2 with respect to e and the 
cancellation e — ^(p/{ri) = e — ilj<l>,i{f2) = 0, the distributional partial derivative of 
g is given by 



de 



(e, r) = 4TT^V2{e - ^j^^r))-^/^ 



and hence the continuity of ri, r2 with respect to e implies the a.e. convergence in 
r of this function when e — >• eo: 



9e 



5e 



eo,r) ase-;>eo, for all r / ri(0, eo, ^), r / r2((/), eo, ■^). (2.24) 



Now from the concavity estimate (2.9): 

C 



0<|(e,0 < 



< 



r^/rlr2 ^ 

VI ^/{r - ri)(r2 - r) ''^ 

C l/lji 1 

VI |eo + Y^(r - ri)(r2 - r) 



'■ri<r<r2 



,(r), (2.25) 



where we applied (2.8) and recall that ri = ri((/), e,£), r2 = r2(i;^, e,^). We observe 
that 

<-^- C l/lii 



,{r)dr 



which implies in particular that G -L^(M+) and gf(e, r) G -^>"'^(/ 



\/£|eo + eP 

dg_i 



(2.26) 
Together 



with (2.24), this implies that Assumption (i) of Lemma |A.l is satisfied. 

Let us now check Assumption (ii). From the continuity of ri((/>, e, tj and r2((/', e, 
with respect to e, we deduce that 

qe{r) ^ qeo{r) as e eo, for all r / ri((/), eo, ^), r / r2(0, eo, £). 
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This a.e. convergence, coupled to the fact that, by (2.26 ), the integral of the positive 



functions is independent of e, is enough to conclude, thanks to the Brezis-Lieb 
theorem (see Theorem 1.9 of [33|), that converges to (/eo in -^"^(1^+) as e — t- cq. 
Assumption (ii) is then satisfied. Hence Lemma A.l can be applied and a^{e, i) is 

with respect to e with its derivative given by (|2.23L From Lemma 2.1 ^{e,£) > 



on ]e<^_£, 0[, so by (2.19 ) e i— )■ a^{e, £) is a diffeomorphism from Je,^^^, 0[ to ]0, +oo[. 
This concludes the proof of Lemma |2.3| □ 



2.2. Regularity properties in of a,^ and a^^. We continue the analysis of 
the Jacobian a^j) and claim further continuity and differentiability properties with 
respect to (j). 

Lemma 2.4 (Continuity properties of a^{e,i) with respect to 0). Let f G Srad 
nonzero, let fn be a bounded sequence in £rad o-i^d denote (/>„ = (j)f^, (j) = (pf. 

-)• +00. Then, for all i > fixed, the 
+oo: 



Assume that V(/>j^ — )■ V0j in L (M'^) as n 
following convergence properties hold as n — 



ri{(f)n,e,i) ^ ri{4i,e,£), r2{ 
infinf(l+r)|(/)„(r)| > 0, 

n r>0 



(2.27) 

,£)^r2(0,e,£), Vee]e^,f,0[, (2.28) 

(2.29) 



a^^(-,£) —7- a^(-,£) uniformly on any compact subset of] — oo,0[, (2.30) 
a^^{s,i) ^ a^^{s,i) foralls>0. (2.31) 

Proof. Step 1. Convergence of the potentials. 

As a standard consequence of interpolation and V(/)n — >• V(/> in L^, we have that 

Pf weakly in L^/^{R^). (2.32) 

Moreover, by Sobolev embeddings and elliptic regularity, together with the spherical 
symmetry of we have: 



and 



(pn^ 4> in L°°(M+) as n +oo (2.33) 
(p' in L~([a, b]) as n ^ +oo, for all < a < 6. (2.34) 



Step 2. Proof of 1^17}. 
To prove that e^h^ i 



e^^i, we first pass to the limit n — )• oo into (2.3), and get 

limsup e<A„,f < e<A ^ (2.35) 



Now, we know that the infimum 
we have 



is attained at r„ = ro( 



and from (2.13) 



2r„ 



\fn 



+ 



2r2' 



We observe that 



inf I/, 



> 0. 



(2.36) 



(2.37) 



Otherwise we would have, up to a subsequence, /n — >■ in L^(M^), and then (pj = 0. 
This means Pf = ^(pf = 0, and then / = 0, which contradicts the assumption 
/ 7^ 0. Therefore, (2.36) and (2.37) ensure that the sequence r„ is bounded and 



bounded away from since (2.35) implies limsup e^^^i < 0. Then, any subsequence 



16 



M. LEMOU, F. MEHATS, AND P. RAPHAEL 



r„/ of r„ satisfies r„/ — )• tq > (up to extraction) and one can pass to the limit in 

e^^,/ = (kn'irn') + ^ to get 



thus 



lim inf 



Finally (2.35) and (2.38) imply (2.27). 



(2.38) 



Step 3. Proof of i\2. 28]) . 

Let e G]e^^£, 0[. From Step 1, we have e G]e0^^£, 0[ for n large enough, thus ri( 
is well-defined. By Lemma [2.1[ ri((/)„,e,£) is characterized by 



(/'n(n) + and (p'nin) 



< 0. 



Moreover, by (2.8), ri{(j)n,e,^) lies in a compact interval of 



(2.39) 
Therefore, after 



extraction of a subsequence, we have ri( 



> 0. Thanks to (2.33) and 



(2.34), one can pass to the limit in (2.39) and obtain 
e = (pir^) + —J and 0'(r*) - 



< 0. 



This is enough to conclude that r^, = ri((/), e,£). We have thus proved (2.28) for ri. 
the proof of r2 (</)«, e, £) — )• r2{(p, e, i) is similar. 

Step 4. Proof of ^29^. 
Let us prove (2.29): 



inf m, 



--: m > 0, 



where m^^ is defined by (2.20). Assume that m = 0. Then there exists a sequence 
r„ such that (1 + r„)(/)„(r„) — )• as n — )• -|-(X). If r„ is bounded, then, up to a 
subsequence, it goes to some vq > and from (2.33), we get 0(ro) = 0, which is not 
possible from (2.11) and / 7^ 0. Hence r„ is not bounded and, up to a subsequence, 
it goes to +00. Let a > 0. We get from (2.12) 



(1 + r„)|0„(r„)| > r„|0„(r„)| > 4tt s Pf„{s)ds > iir s Pf„is)ds 



for n large enough. Passing to the limit in this inequality and using (2.32), we get 
Pf = 0, which again contradicts the assumption / 7^ 0. We have thus proved (2.29). 



Step 5. Proof of [2.30). 



Now, we prove the uniform convergence of a,^^. We observe from (2.8 



terval of integration in the expression (2.18 ) of a^^ is bounded. Thus, t 
convergence theorem applies since 



that the in- 
le dominated 



Mr) 



2^ 



1/2 



from (2.33). This yields a0„(e,^) — t- a^{e,£), for all e < 0, £ > 0. Now using the 
monotonicity of the function e — )• a^^{e, t) at fixed £ and applying the second Dini's 
theorem, we get the desired uniform convergence. 
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Step 6. Proof of 



Let {s,t) £ {R\y. Denote 



We will prove that e„ — t- eo- From (2.21 ), we get 

2 



l/nl 



l/nl 



Now we claim that 



\eJ > C 



m 



> if I e„ I < 



m 



A{£ + 2m) ■ 



Indeed, we first get from (2.22) 



\eJ > C 



m, 



4>n 



>0, 



(2.40) 



(2.41) 



(2.42) 



provided that |e| < g^jqi^^^j with m^^ defined by (2.20). From (2.29), we have 



'>T^(j>„ > m > 0. Therefore, (2.42) implies (2.41) since the function t i— )• 
increasing. 



IS 



We then deduce from (2.40) and (2.41 ) that the sequence belongs to a compact 
interval of W_ thus, up to a subsequence, we have e^i — y ^ 



as n — )• +00. 



Using (2.30), we have 



s = a^rii^^n,^) ^ a^{eoo,£) as n +oo. 



Hence, 



(coo,-^) = a^{eoJ) = s e (0,oo). 
Since e i— )• atf,{e,i) is invertible from (e0^£,O) onto (0, oo), we deduce that 

eo = CLT^is,^^) = Coo, 



which means that Cn — cq as n — t- +oo. The proof of (2.31 ) is complete. 
This concludes the proof of Lemma |2.4| 



□ 

Let us now examine the differentiability of and a^^ with respect to (f). To 
shorten the statement of the next lemma, we introduce a few notations. We consider 
two nonzero potentials (j) = (f)f £ ^rad and (j) = £ ^rad and set: 



h = (j) — 4). 

For all £ > and A G [0, 1], we recall the notation 

e<i,+\h,i = inf {il}ci,/{r) + \h{r)) , 

r>0 



where ip(j)Q/{r) is defined by (2.2), and denote 



n{(l), (j), £) = {{X,e): AG [0, 1] and e e]e^+xh,i,0[} ■ 



(2.43) 
(2.44) 

(2.45) 



Let s £ M*^ and A G [0,1]. Recall that, by Lemma 2.8 there exists a unique 
e £]ecf,+xh,e,0[, denoted by a^l^^{s,£), such that a^+xh{e,i) = s. Finally, we set 



M = max(|/|ii,|/|ii). 



(2.46) 
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Lemma 2.5 (Differentiability of a^{e^t) with respect to (p). 

Let i > be fixed. Consider (f),(f) €z $rad both nonzero and let h = (p — (j) . Then, with 
the notations (2.43)-(2.46), the following holds: 
(i) The function 

(A,e) H> a<^+A/,(e,^) 
is a function on il,{(p, cj), £). Moreover, we have 



d_ 
dX 



(e,£) = -i7r^V2 



(e - V'^/W - A/i(r))-i/2 h{r)dr, (2.47) 



with the bound: 



dX 



ie,£) 



< C- 



V(A,e) G ^{<l),(^,l), 



(2.48) 



for some universal constant C > 0. 

(ii) Let s € M!^. Then the function A i— ?• 0'~^\\h{s,£) is difjerentiable on [0, 1] and we 
have 



A -1 ( 



( -1 \-l/2 

\^4,^\hk^^ ^) - V'</./(r) - Xh(r) j h{r)dr 



-1/2 



(2.49) 



where (rj),=i^2 shortly denotes ri{(l) + Xh,a^^^j^{s, £),£). 



Proof. Recall from Lemma 2.4 that the functions e^^\h^£, ri{(f) + Xh, e, £) and r2{4> + 
Xh,e,£) are continuous functions of A (for fixed e and £). 

Step 1. Proof of (i). 



This proof of (i) will be done with Lemma A.l, exactly in the same manner as the 
regularity of a^+A/i(e, £) with respect to e in Lemma 2.8 We fix £ > and introduce 
the following function 



so that 



giX, e, r) = 87r2^/2 (e - ^P^^r) - Xh{r))]/^ ^ 

rr2{<j>+\h,e,£) 



a<p+\h{e,£) 



ri{(t)+Xh,e,e) 



g{X,e,r)dr. 



By (2.13) and (2.8), we have the following uniform bound: 

M 



g{X,e,r)<C 



ri{(P + Xh,e,£)J -^71' 



where M is defined by (2.46). Hence, one deduces from standard dominated con- 
vergence that (A, e) i— )• a(j)+\h{e,£) is a function on [0, 1] x M_ and satisfies 

a0+Ah(e, £) > 4^ (A, e) G Jl((/>, p), £). 

Let us now prove the differentiability of a^p^xhie, £) with respect to A. Let Aq £ 
[0,1], Co = e^+Ao?t/, and e s]eo,0[ be fixed. From the continuity of e^+A/i,^ with 
respect to A, we have e £]e^^xh^£,0[ for A in a neighborhood Iq of Aq- Hence, for 
X £ Iq, the distributional partial derivative of g is given by 

^(A,e,r) = -47r2\/2(e - ip^/ir) - A/i(r))"^/^ lri(</.+Ah,e,^)<r<r2{</.+A?t,e/) ^('^)- 
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Moreover, from the continuity of ri((/) + A/i, e, i) and r2{(p + Xh, e, i) with respect to 
A, we get that 

^^-(A,e,r) ^ If (Ao,e,r) as A ^ Aq, (2.50) 



dX 



dX 



for all r / ri((/) + Xoh,e,£), r / r2((^ + Xoh,e,i). Now, we use (2.9) and (2.8): 



do 

0< ^(A,e,r) <C 



dX 



\^ V^(r - ri)(r2 - r) 



<r<r2 



with 



^2 

qxAr)dr = C—-='K. 



(2.51) 
(2.52) 



As in Step 2 of the proof of Lemma 2.8 one deduces from (2.50), (2.51), (2.52) 



and from the Brezis-Lieb theorem that Assumptions (i) and (ii) of Lemma A.l 



are 



satisfied. Hence the function a^+A/i(e,£) is differentiable with respect to A and its 
differential -^a^j^\h{e,t} is given by (2.47). 

We now claim that -^a(j,^\h{e, i) is a continuous function of (A, e) on 0((/), 0, £). 
Indeed, let Aq G [0, 1] and eo = e^+\oh,t be fixed. A direct adaptation of the proof 
of (2.28) enables to show that 

ri(</) + Xh, e,£) —?■ ri{(f) + Aq/i, eo,i), r2{4> + Xh, e,i) r2{(t> + Aq/i, eo,£) 
as (A, e) — )■ (Ao,eo). Thus we have the following a.e. convergence: 
dg 



dX 



(A,e,r) 



dX 



(Ao,eo,r) as (A, e) (Aq, eo), for all r / r2((/) + Aq/i, eo, ^). 

(2.53) 



Hence, using again the domination (2.51 ) and the fact that qx f> — )• Qx^^eo i^i 
as (A, e) — )■ (Aq, eo), we deduce from dominated convergence theorem that 



+00 



(A, e, r)dr 





In other words 



dg_ 
dX 



+ 00 



dX 



(Ao, eo, r)dr as (A, e) (Aq, eo). 



Xh{e, £) is a continuous function of (A, e). Similarly, ^a(f,^\h{e, I) 
is a continuous function of (A,e). Therefore, the function (A, e) i— t- a^^\h{e, i) is 
on ((/),(/), £). Since the bound (2.48) stems directly from (2.51) and (2.52), the 
proof of Item (i) oi Lemma 2.5 is complete. 

Step 2. Dijferentiability of a'^^^f^{s,£). 

Let {s,i) G (M;^)^. From Lemma 2.4 we already know that the function A i— t- 
'^0+A/i('^' ^) continuous. Let Ao G [0,1] and consider a sequence A„ G [0,1] such 
that A,i — )• Ao as n — )• +oo. We write 



where we have set 

AliXn 



and 



A2{Xr, 



, , Ai{Xn)A2{Xn), 

An — Ao 

°^+A„fe(-'^^^) -«^+Aofe(^'^) 

a<^+Ao/.(a^+A„h(s,^),^) - a<p+Xohia^lxoh(^^^)'^) 

a(^+Xoh{a^lx„his, «</.+Aofe(a0+Aofe(^' ^) 

An — Ao 

Q<^+Aofe(a^+A„fe(^' ^) - «</-+Anfe(Q0+A„fe(g' ^) 
An — Ao 



(2.54) 
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and where we simply used that a^+Xoh{a^lxoh(^^ ^) = s = a^+\„h{%lx„hi^^ ^) 



Let us examine separately the convergence of the two factors Ai and A2 in ( |2.54 ) 
From the continuity of A 1— t- o,'^^^^h{s,i), we have: 



(2.55) 



hence 



lim Ai{\r, 



da 



de 



47r2\/2 



Xoh{r] 



-1/2 



(2.56) 



dr 



Let us now examine the convergence of the term A2{Xn), that we rewrite as follows: 



^2 (An) 

where we have denoted 



An — A 



J\o 



OA 
5A 



(/i, en)d/i, 



A{X,e) = a0+Ah(e, 



a 



-1 

(f>+X„h 



s,£). 



By (|2.55|), we have Cn eo = a^_^^^f^{s, 



Moreover, from Step 1, we know that 



the function (A, e) 1— )■ §4(A,e) is continuous at (Ao,eo). Hence, we have 



dA OA 

^"^f = ^(Ao, eo) 

/i— s>Ao, ra— s>oo OA OA 



/ A=Ao 



from which we deduce that 



lim A2(An) 

n— >+oo 



-Xh 



dX 



X=Xo 



«0+Ao/.(«'^)'^) 



47r2\/2 



r2 



ri 



^) - "^0/(0 - XQh{r) ) h{r)dr. 



"1/2 



where we used (2.47). Finally, (2.56) and (2.57) give ( 2.49^ . This concludes the 
proof of Lemma 2.5 



□ 



2.3. Rearrangement with respect to a given microscopic energy. In this 
section, we introduce the Schwarz symmetrization of a function / with respect to a 
given microscopic energy e = — h </>(x) at given momentum ^ > 0. 

We start by defining a suitable rearrangement of / G 8rad given momentum I > 
which preserves the generalized Casimir functionals ( |1.9[ ). We proceed similarly 
like for the usual Schwarz symmetrization, see |33 | I27 | [T|. Let us recall the definition 
(1.13), (1.14) of the distribution function of / at given £ > 0: 



i^i {(r, u) G Qi, f{r, u, I) > s} 

hoo f+00 



r=0 Ju=—oo 



We then have the following elementary lemma: 

Lemma 2.6 (Properties of fxj). Let f ^ nL°°(M^), nonnegative and spherically 
symmetric, and let fj,f{s,£) be the distribution function of f at given i as defined by 
(1.14). Then there exists a set A with \A\^+ = such that 



GM+\A, Vs>0, Hf{s,i)< +00, 



(2.57) 
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M£€R+\A, Vs>|/|loo, Hf(s,£) = 0. (2.58) 
Moreover, Mi E M+\74, the map s — )• fj,f{s,i) is right continuous on M^. 

We may now introduce the generalized Schwarz symmetrization: 

Proposition 2.7 (Schwarz symmetrization at fixed £ > 0). Let f £ H L°^(M^), 
nonnegative and spherically symmetric, let ij,f{t,£) given by (1.14) and let A be the 
zero measure set given by Lemma 2.6 We define the Schwarz symmetrization f*{-,i) 
of f at fixed i as being the pseudo inverse of fif {■,£): 

sup{s > : fif{s,£)>t} for t < fif{0,£) 

for t> HfiOJ) 

(2.59) 

with fif given by (1.14). Then f*{-,£) is a nonincreasing function on [0,oo) and 

Mt>0, M£eRX\A, fif{t,£) = \{s>0;f*{s,£) >t}\^+ (2.60) 

In particular 

I/*Ilp(r+xR+) = I/Ilp(ir6), VpE[l,+oo]. (2.61) 
Moreover, there holds the contractivity relation: 

ir-fflii < (2.62) 



Vt>0, M£eRl\A, f*{t,£) 



Lemma |2.6| and Proposition |2.7| can be derived from standard arguments by 
adapting for example the arguments in jH], this is left to the reader. 



Given / E £rad and (p E ^rad^ we now define the rearrangement of / with respect 

I |2 

to the microscopic energy — h 0(x). 

Proposition 2.8 (Symmetric rearrangement with respect to a given microscopic 
energy) . 

Let f E S rad fJ-i^d (j) E ^rad i^on zcro. Let f* be its symmetric rearrangement defined 
by (2.59). We define the rearrangement f*^ of f with respect to the microscopic 



\ |2 

energy '-^ — h (/>(x) by: 

f*^{x, v) = r (a^ + 0(x), \x X v\^^ , 

where is defined in Lemma 2.3, Then 

G Eq(/) 



X X ti 



■J^+</.(x)<0 



where Eq(/) is defined by (|1.15 ), and in particular: 



(2.63) 

(2.64) 
(2.65) 
(2.66) 



\f*%v = \f\Lv, VpE[l,+oo]. 
Moreover, f*'f' E £rad with estimate: 

Before proving this proposition, we give a corollary which says that nonincreasing 
steady states are invariant through the above rearrangement. 

Corollary 2.9 (Identifi cation of Q*'^Q). Let Q satisfy Assumption (A ) and let Q*'t'Q 
be defined according to (2.63). Then Q*'^'^ = Q and we have 



Q* {a^Q{e,£),£) =F{e,£), M£ > 0, Ve E [e^^,,, 0[, (2.67) 

In particular, for all £ > 0, Q*(-,£) is a 



2.1 



where e^g^£ is defined in Lemma 
function on ]0, fj,Q{0,£)[, where fig is defined by (1.13) 
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Proof of Corollary 2.9 Let £ > be fixed and recall the function F defined in 



Assumption (A). Assume first that F{e,i) = for all e < 0. From definition (1.13) 



we have f^Q{s, i) 



|(r,' 



F 



for all s > 0. This 



implies from (|2.59|) that Q*{-,t) 

is not zero on 



0, and then identity (2.67) is satisfied, 
and let 



Assume now that F{-, 

eo(^) = sup{e < : F(e,£) > 0} . (2.68) 

By Assumption (A), we have eo(^) < cq < and the function e i— )• F(e,i) is 
continuous, strictly decreasing on ] — oo,eo(^)] and vanishes for e > eo(^). As F is 



nonnegative, we have from (|1.13|): 
fiQ{F{e,£),e) = i^e 



\u\' 



+ 



r),£j > F{e,£)j , Ve G 
oo,eo(^)], this identity implies 



(r, u) : F 

and, F(-,i) being strictly decreasing on 

fiQ{F{e,i),£) = z/^|(r,n) : ^ + (/>Q(r) <e|, Ve < eoW 

= a^Q{e,£), Ve<eoW. (2.69) 

Assume that fiQ{0,£) = 0, then /xq(-,^) = since it is a nonincreasing function. 
Hence, from definition (2.59) we get Q*{-,£) = 0. Now, we write (2.69) for e = eo(^) 
and deduce from the structure of a^^ that eo{£) < e(f>Q,i- This means that i^(e, £) = Q 
for e G [e(/)Q,^,0[, and identity (2.67) is satisfied. 



We now assume iiq{0,£) > 0, w 



Tiich implies from (2.69) that eo{£) > e^g^£. We 
know that afj,g{-,i) (resp. F(-,i)) is continuous and one-to-one from [e(j,Q/,eQ{£)] to 
[0,a^g{eo{i),£)] (resp. [0, F{e^Q^(:, i)]). Hence, identity (2.69) ensures that ^q{-,£) 
is invertible from [0, -F(e0g^£, £)] to [0, a^^g (eo(^), ^)] and Q* (which is by definition 
its pseudoinverse) is its inverse in this case. Therefore, (2.69) implies 



Q*[a^g{e,£),£)=F{e,£), Ve e [e,^^,,, eo^]. 



Now (2.69) implies that a(^g(e,^) > a0p(eo(^),^) = ^q{0,£) for e G [eo(-^),0[, which 
together with the definition of Q* ensure that both terms in (2.67) vanish for e G 
[eo(^),0[. This ends the proof of ( 2.67[ ). Finally, using (2.67), we conclude that the 
stated regularity of Q* on ]0,a(j,Q{eo{£),£)[ is an immediate consequence of the 

regularity and the non vanishing derivatives of F and aif,^ on ]e^g^£,eo{£)[. 

To end the proof of Corollary 2.9 , it remains to identify Q and Q*'I'Q for a.e. 



\X X v\ 



> and let e{x, v) 



+ 



r) > 



< 0, then (2.67) gives 



X, V. Let (x, v) G such that I 

'0</>Q,^(^) ^ ^<t>Q.i^ where we used that |fp > If e{x,v 

directly Q{x,v) = F{e{x,v),£) = Q*'t'Q{x,v), by Assumption (A) and ( |2.63[ ). If 
e{x,v) > 0, then we have Q{x,v) = F{e{x,v),£) = Q*'^'^{x,v) 
( |2.63 ). This concludes the proof of Corollary 2.9 



0, using again 
□ 



Proof of Proposition 2.8 



We first notice that the formula (2.63) is well-defined for a.e. {x,v) G by Propo- 
sition 



2.7 Indeed, from ( |1.10 ) we have that 

\{{x,v) G 



X X v\ 



GA}\ 



0, 



where A is the measure zero exceptional set given in Lemma 2.6 



Step 1. The change of variables formula. 
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The equimeasurability of / and f*'^ relies on the following elementary change of 
variables formula: let two nonnegative functions a £ C''(M) n L°°(M), f3 G L-'^(M+ x 
M+), then > 0, 



oo r+oo 



a 



u 



'+(/)(r)<0 







(2.70) 



where is given by (1.12). This implies in particular from (1.10): 

,|2 



a 



+ (f>{x) j /3 I I — — h (j){x), \x X v\ 1 , |a; X J dxdv 



2 

-oo /■+00 



a{a^\s,e))P{s,£)dsdt 



(2.71) 



Let us pro ve (|2.70 ) . We first perform the change of variable on the integral on u in 
the Ihs of (2.70), e = ^ + 0(r), to obtain 



a 



r,u>0 



U 



+00 pO 



a 



r=0 J e=—oo 



(e)/3(a^(e,^),^) e-,/.(r) 



2 +</.(r)<0 
-1/2 



2r2 



1 



dedr. 



Now from Fubini and (|2.23), we get 

2 1 a{ — + (t>{r)] /3 ( 



r,u>0 







a(e)/3 {a<i,{e,£),e) —[e.ijae. 



2.3 



for any ^ > 0, the map e i— t- a(j,{e,£) is a C^-diffeomorphism from 
, and we may therefore perform the change of variable s = a^{e, i), 



From Lemma 
]e</,,£,0[to]0, +00 
which together with (|2.19[) yields (|2.70[). 



Step 2. Equimeasurability and proof of (2.66). 



Let now cf) G <^rad, f e Srad a nd f* t> given by ( |2.63| ). We first prove that f*"' G Eq(/) 
according t o the definition ( L_15 ). For all t > and i > 0, we use the definition 
of Hf{-,i) ( |1.14| , of f*'^ ( |2.63[ ) and the formula ( |2.70| with a = 1 and /3(s,£) = 
lr(5/)>t to get: 



fif*4t,£) = 2 



oo /"+00 



1 



f*'l>{r,u,e)>t 



dui 



+00 



and hence from (2.60): 



Vt > 0, a.e. £>Q, iJ,f*4.{t,£) = fif{t,£), 



which implies the equimeasurability of / and /*''' according t o the definition (1.15). 
It remains to control the kinetic energy of f*'f' according to (2.66). Indeed: 

\\v\'^f*^\Li = 2 j + f*'^{x,v)dxdv + 2 j V<^{x)-V(t)f,^dx 
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where we used (2.63) and the interpolation inequahty (2.10). This together with a 
straightforward locahzation argument concludes the proof of (2.66). 



This concludes the proof of Proposition 2.8 



□ 

Let us conclude this section with an elementary Lemma which will be useful in 
the sequel. 

Lemma 2.10 (Pseudo inverse of f*{a^{-,l),€)). Let f G S^ad CL^d (j) G ^rad be 
given nonzero functions, and let i > such that f*{0,i) > 0. The function e i— )• 
f*{a(f,{e,i),i) is nonincreasing from [e^^i,Q[ to [0,/*(0,£)]. We define its pseudo 
inverse, which we denote (with abuse of notation) s — )• (/* o a^)~^(s,£), as follows: 



(r o a^r\s,e) = sup{e G [e^,,,0[: f*{a^{e,£),e) > s}, 



(2.72) 



for all s £]0, f*{0,i)[. Then s — t- (/* o a^) ^{s,£) is a nonincreasing function and 



V(x,w) G (I 



&^)^ such that \x x 

f*Hx,v)>s = 



i, vs e]o,f*{o,e)[. 



+ <Pix) < if*oa^)-'is,i), 



(2.73) 



f*Hx,v)<s 



+ Hx) > {f*oa^r\sJ). 



(2.74) 



Proof. Let £ > and s G (0,/*(0,£)), then f*{a^{e^^i4),t) = f*iO,i) > s and 
hence 

{e G [e^,e,0) : f* {a^{e, i), I) > s} is not empty. 

This means that (/* o aif,)~^{s, £) is well defined for s G (0,/*(0,^)). The mono- 
tonicity of (/* o a^)~^ follows from the monotonicity of /* and a,^. 

Let now {x,v) G be such that \x x up = £ > 0. Assume f*'^{x,v) > s, 
then /*(o^( 

./,0), and 



bp 



+ 



(x) < 



f, and in this case (|2.73| is trivial, or + 



this implies 



+ 



;) < if* ° a^)~HsJ)) from the definition (2.72). Thus (2.73) 



is then proved. Assume now f*1'{x, v) <s. If ^ + (pix) > then ( |2.74| is trivial, 

0). Thus for all 

+ (f>{x) > e, 
□ 



x) G 



+ 4>(x) G 



otherwise f*'^{x,v) < s < f*{0,£) implies that 

e G {e G [e0^£,O) : f* [a^{e,£),£) > s} which is a non empty set, ^ 
and (|2.74|) follows. 



3. Nonlinear stability of the Vlasov Poisson system 

This section is devoted to the proof of the main results of this paper. We first 
exhibit the key monotonicity formula involving the generalized symmetric rearrange- 



ment with respect to the Poisson field (2.63), Proposition 3.1, which allows us to 



reduce the study of the minimization problem of Theorem 1.3 to the one of an 
unconstrained minimization problem on the Poisson field only. The study of this 



new problem, that is the proof of Proposition 3.2 is postponed to section |4j and 
immediately yields Theorem 1.3 We then show how to extract compactness from 



minimizing sequences to prove Theorem 1.4 which now implies Theorem 1.5 from 
standard arguments. 
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3.1. The monotonicity formula. Given / G £rad^ we will note to ease notation: 

/ = r<^/, (3.1) 



and recall from Proposition 2.8 that: 

/Gf,,dnEq(/). 
We introduce the functional of </> G ^rad'- 

Jr{<l>) = nr'i') + hvci>-VcPj 



(3.2) 



(3.3) 



and claim the following monotonicity formula which is a fundamental key for our 
analysis -see also [2j for related statements-: 

Proposition 3.1 (Monotonicity of the Hamiltonian under the f*'^f rearrangement). 
Let f € Srad, non zero, and f given by (3.1), then: 



n{f)>jf^{<Pf)>nf)- 

Moreover, T-L{f) = T~L{f) if and only if f = f- 
Proof Let f,g e £rad, then: 



(3.4) 



nf) 



\v\'f 



1 



n{g) + 



+ 



{f-9) 



V0/ • V^g 



and hence the general formula: V/, 5 G ^radi 



n{f) = n{g) + ^|V</./ - VcPgll, + (^M! + ^^(^)^ (/ _ g) dxdv. (3.5) 
We apply this formula with g = f = f*'^f and rewrite the result using (3.3): 

,|2 



+ 0/(x) if -f) dxdv. 



We now claim: 



JR6 V 



+ <Pf{x) {f-f)dxdv>0, 



(3.6) 



with equality if and only if / = /, which immediately implies (3.4). 

The proof of (3.6) is reminiscent from the standard inequality for symmetric 
rearrangement known as the "bathtub" principle 

/ \x\f* > [ \x\f, 
see |33) . Indeed, use f(x,v) = lf^j(^j.,y^dt and Fubini to derive: 



dt 



t=o 

00 

I 

£=0 



^f] if - f) (^^dv 
2 



+00 



dt 



t=o 



+ cPf [1 



1, 



f{x,v)<t<f{x,v) '^f{x,v)<t<f{x,v) 



dxdv 



dxdv 



no/) 



dt 



t=0 



+ (l)f{r) dui 



(3.7) 
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where dvi is given by (1.12), and 

SiAt) = {ir,u) G neJ{r,u,£) < t < f{r,u,£)}, 



S2,i{t) = {{r,u) G n,J{r,u,l) < t < f{r,u,£)}, 



We now use ( |2.73[ ) in Lemma [27L0| to obtain: Vt E (0,/*(0,^)), 

,2 



where we recall that 



u 



+ 0/(r) di^e < if* o a^)-'{t,i)i^iiS2At)) 



Js2,i(t) 

We then observe from / G Eq(/) that: 

for a.e. t > 0, v^{SiAt)) = ut{S2/{t)), 

and deduce 



it) 



dvi. 



Injecting this into (3.7) and using (2.74) yields: 

,|2 



+ (t}f]{f - f)dxdv> 



00 j-riQ/) 

d£ / dt 



=0 Jt=0 



SiAt) 



u 



and the analogous inequality for S2/{t): 



+ (Pf]{f - f)dxdv 



> 



u 



00 rf*{o,e) r f 

d£\ dt ({roa^)-\t,£)-^-(l)f{r)\diye>0. 

=0 Jt=o .<f'^ \ 2 



Moreover, assume that Jj^e ^^-5 — \- (j)f{x)^ (/ — f)dxdv = 0. Recalling that 

f^(S'i^£(t)) = t'£(5'2/(t)) = for t > f*{0,£), the above two chains of equalities 
imply that for a.e t,£ > 0, either lye^Si^t)) = i^e{S2/{t)) = or i^e^Si^t)) = 
MS2Ai)) > with: 

f + = (r o a^)-\t, ^) = f + ./'/(rs), 

for a.e (ri,'Ui) G a.e (r2,n2) G S2.eit), which contradicts the fact that 

f{ri,ui,£) < t < f{r2,U2,£)- We conclude that a.e t,£ > 0, i'e{Si^i{t)) = 
i'e{S2/{t)) = which implies f = f- This concludes the proof of (3.6) and of 
Proposition |3.1[ □ 
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3.2. Reduction to a variational problem on (j) and proof of Theorem 1.3[ 

We now claim the following local coercivity property of the functional of (p given by 
(3.3). To ease notations, we let for (j) G ^rad'- 

J{<P) = jQ'{(t^)=nQ*'^) + \( |V</)- V(^Q.,p (3.8) 



Proposition 3.2 (0q is a local strict minimizer of J). 

There exist a constant Co > such that the following holds. For all R > 0, there 
exists 6o{R) g]0, ^|V0q|^2] such that, for all f G S^ad satisfying 

\f-Q\e<R, \V4>f-V4>Q\L^<So{R), 



we have 



J(.ct>f)-J(.^Q)>Co\V^f-Vcl)Q\ 



L2 



(3.9) 



The proof of this Proposition essentially relies on Antonov's coercivity property 
and is postponed to section [4] Theorem |1.3| is now a straightforward consequence 
of Propositions |3.1| and |3.2[ 



Proof of Theorem 1.3 



Let R > and / E Srad H Eq((5) satisfying (1.18), where do{R) is as in Proposition 



3.2 In particular, note that 



implies that (pf ^ and / 7^ 0. Then the monotonicity property (3.4), /* = Q* and 



(3.3) yield: 



n{f)-n{Q) > Jf*{^f)-n{Q) = j{^f)-nQ)- (3.io) 

On the other hand, recall from Corollary |2.9| that our assumption on the ground 
state Q ensures 

Q = g*<^Q=Q and thus n{Q) = J{cPq). 
Injecting this together with (3.9) into ( 3.10| ) yields: 



nf) - n{Q) > JicPf) - JicPg) > Co|V0/ - V0q| 



L2, 



(3.11) 



this is (1.19). If in addition T-L{f) = H{Q), then (pf = (f)Q and hence using /* = Q*: 

'H{f*^i) = 'H{Q*^f) = niQ*'''^) = niQ) = nif). 

We thus are in the case of equality of Proposition |3.1| from which: 

y _ J*(pf — p4>Q — Q*4>Q — Q 



This concludes the proof of Theorem 1.3 



3.3. Compactness of minimizing sequences. We are now in position to prove 
Theorem 11.41 



Proof of Theorem 1.4 



The key to extract compactness is the monotonicity formula (3.11) which yields a 



lower bound on the Hamiltonian involving the Poisson field (pj only, while standard 
Sobolev embeddings ensure that (pf enjoys nice compactness properties in the radial 
setting. 



Step 1. Weak convergence in U' , p > \. 
Let 



R = \Q\s + C{1 + |V0qU2)^/3 |Q|[/9 \Qf/l + IQUi + \Q\l. 



(3.12) 
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where C is the constant in the interpolation inequahty (2.66) 



Let /„ G £rad t>e a sequence satisfying ( 1.21 ), ( 1.22 ), where 6 will be fixed further, 
satisfying in particular 

(3.13) 



5 < min { 1, -|V(/)Q|i2 



Observe that (1.21) and (3.13) imply (j)f ^ 0. The sequence /* is bounded in 



by (1.22), so fn is itself bounded in L^. More over, from ^(/n) < C, the L°° bound 
of fn and the interpolation inequality (2.10), |t;p/n is uniformly bounded in L^. 



Hence fn is bounded in the energy space Srad- We then get: 

fn^f^ Srad in L'^ for all 1 < p < +oo, 



(3.14) 



up to a subsequence. Moreover, by a standard consequence of interpolation, Sobolev 
embeddings and elliptic regularity, we have 



I W/n - V(/)/|i2 and - (/)/|loo -> 
From assumptions (1.21) and ( |1.22 ): 

|V</./- V0Q|i2 < ,5. 



as n — )• +00. 



(3.15) 
(3.16) 



In particular, (j)f ^ 0, since 5 < \V(Pq\£^2 from (3.13). Hence, by Proposition 2.8 
we have 

Q**/GEq(Q). (3.17) 



Step 2. Strong convergence in £ of the sequence Q*'^f" . 

We now aim at extracting a preliminary compactness from /„. Let 

fn = Q*'^f-, f = Q*^^, 



(3.18) 



and observe that fn is in fact a function of (pf^. We then claim that the strong 
convergence (3.15) automatically implies some strong compactness in £ for /„: 

(1 + bP)7n ^ (1 + bl')7in L1(M6). (3.19) 
We claim also that there exists 5i{R) such that, for < 5 < Si{R) we have 



\V(t>J-V(t>Q\L2 



< 



(3.20) 



where R is defined by (3.12 ) and 6q{R) is defined in Theorem 1.3 We are now ready 



to fix the constant S of Theorem 11.41 as follows: 

1, 



6 = mm{l,^\V^Q\L2,6,{R)]. 



Proof of {3.19), (3.20). We first claim the a.e convergence: 



fn—^f as n +00 for a.e {x,v) eM.^. (3.21) 
Indeed, let (x, v) G such that \x x v\'^ = £ > 0. If e = ^ + (j)f{x) < 0, then from 

I I |2 

(3.15), — h (f)f^^{x) < e/2 for n large enough and 



/„(x)<-0;„(o)<a 



We now recall from Lemma |2.4| that for all £ > 0: 

a<f>f„{e,£) a^f{e,£), 



(3.22) 
(3.23) 
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uniformly with respect to e lying in a compact subset of ] — oo,0[. Therefore, from 



+ (t)n{x) < e/2 < and from (3.22), 



+ (t>f„{x), \x X v\ 



+ 4>f{x), \x X 



as n — 7- +00. 



Since, by Corollary 2.9 Lemma 2.3 and Assumption (A), the function Q*{-,i) is 
continuous, this implies Q*'^fn {x,v) — )■ Q*'^{x,v). Similarly, J^+(/>j(x) > implies 

— I" ^n{x) > for n large enough and thus Q*'f'fr^ [x,v) = Q*'^f{x,v) = 0. Hence 
Q*<^/n Q**f a.e in and (|3 2l| is proved. 

Now recall from Proposition 2.8 and from (pf^ 7^ 0, 0/ 7^ 0, that /„ G Eq((5) and 
/ e Eq((5) so that 



Vn > 1, 



fn 



Q 



f. 



The almost everywhere convergence of /„ = Q*'^fn to / and the fact that l/nl^i = 
Iflii allows us to apply the Brezis-Lieb Lemma (see [33], Theorem 1.9) and get the 
strong convergence 

fn^f in as n +00. (3.24) 
It remains to prove the strong convergence of the kinetic energy. Let us decompose 

fn = l|t,|2<i?/n + l|„|2>^/„ = gn,R + hn,R- 

The convergence (3.24) implies: Vi? > 0, \v\'^gn,R — )• |f pl|t,|2<^/„ in L^. 
Consider the other term. We recall that fn = Q*'^-^" is supported in the set 

I 1 2 

~^ — ^ < 0- Hence, by interpolation. 



\vVh 



1^1 hn,R{x,v) dxdv < — 2 / ((>f^{x)hn^R{x,v) dxdv 



,7/12 1^,1/6 



which yields 
By writing 



\v\'^hn,R\L^ ^ C |^n,_R|[(^- 



\hn,R\L^ < \Q*^f" -Q*'^^L^ + [ Q*'t'f{x,v)dxdv, 

J\vP>R 



we obtain that | |^'p/ln,_R|Ll converges to when R — t- +00 and n — )• +00 inde- 
pendently. This together with the convergence of \v\'^gn,R concludes the proof of 
(3l9l). 



We now turn to the proof of (3.20) and claim that it follows directly from (3.16) 
and the definition / = Q*'^f . Indeed, arguing by contradiction, we extract a subse- 
quence V(/)„ V(/>Q in L2 and Qn = Q*"^" such that |V(/)g„ - V(/>Q|i2 > ^p-. From 
(2.66), Tjn is a bounded sequence in 8^.ad and then the same proof like for (3.19) 
yields 

(1 + \v\'^)gn ^ (1 + bP)Q*'^« = (1 + bP)Q in 
and hence V^^^^ — )• V^q in L^, a contradiction. This concludes the proof of (3.20). 

Step 3. Identification of the limit. 



Following (3.1), we let: 



7 — f *<t'fn 
Jn — Jn 



(3.25) 
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We now claim that the variational characterization of Q given by Theorem 1.3 and 

(3.26) 



the monotonicity of Proposition 3.1 allow us to identify the limit: 

7= Q and (^^- = 0/ = (/)q, 
and to obtain the additional convergence: 



+ (t'fni.x) ifn - fn) dxdv -^0 as n +00. 



(3.27) 



Proof of Is. 26^ , {3.27^ . First observe from ( |3.19[ ), = |Q*|l°° and ( |2.10[ ) that: 

n{Jn)^n{J), V^j^^V^j mL\ (3.28) 



From (2.71), there holds: 



l/n /hIlI 



l/n - Q*\ \ Hfn 

+00 r+00 

/ \f*n-Q*\[sJ)dsM 

'0 Jo 

as n — 7- +00 



\x X ) , |x X ) 1 1„|2 



from the assumption (1.22). Together with ( 3.19^ , this yields: 

/„^7in L1(M6). 



(3.29) 



We now invoke the identity (3.5) with g = /n to derive: 

|2 



^\V(t>f„ - vc^jf + n{fn) - nQ) + 



nfn)-n{Q) + 



v\ 



+ 4>fAx) ifn- fn) dxdv 



+ (t>u(.^) I ifn - fn) dxdv. 



Let us examine the various terms of this identity. From (3.28) and (3.20) 

|V0j^ - V0Q|i2 < 5o{R) 



(3.30) 



(3.31) 



for n large enough, where So{R) is de fined in Theorem 1.3 Moreover, from the 
definition (3.18) and from Proposition 2.8 we have the following estimates on fn 
and /: 

|7|li = IQIli, \f\L^ = \Q\L^, \\v\'f\L^<C\Vcl,f\%^\Qf^,'\Q\];i, 



where C is the constant in the interpolation inequality (2.66). Since, by (3.13) and 



( 1.21 ), we have 



we deduce from ( |3.15 ) and ( 3.12[ ) that 

\fn-Q\£<\Q\£ + \fn\£<R, \f-Q\£<\Q\£ + \f\e<R- (3.32) 



Therefore, from (3.31 
Q given by Theorem 



, (3.32) and /„ G Eq((5), the variational characterization of 



.3 



ensures: 

n{fn)-n{Q)>o. 



Next, from (3.6): 



+ (Pfn{x) {fn- fn)dxdv>0. 
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We now claim that: 



+ (f^fn ] ifn - fn) dxdv as n +00. 



(3.33) 



Indeed, from (2.71): 

,|2 



+ (a;) Un- fn)dxdv 



+ {Q* - fn) (a<Pf^ + ^uix), k X 



Xl |„|2 



X X v\ 
dxdv 



+00 



%l/(^)(Q*-/n)(s>^) dsdi, 

where we recall that a^^ is the inverse of the diffeomorphism e — )• 0^(6, £). From 
(2.5), and (3.15), we have \%^^^^is)\ < -e^^^/ < -</'/„(0) < C, and 



Lemma 



2.3 



hence the convergence of /* to Q* yields (3.33). 

Finally, since (1.22) gives lim sup'H{fn) < T-L{Q), we deduce that all the nonneg- 

n— >+oo 

ative quantities in the left-hand side of ( |3.30 ) converge to 0: 

V0/„ - V(/>j^^O in L2, (3.34) 

nln) ^ niQ), (3.35) 



and (3.27) holds, and moreover: 



(3.36) 



Hence, ( |3.28[ ) and ( |3.35[ ) imply n{f) = n{Q) and thus ( |3.20[ ), ( |3.32[ ) and Theorem 
11.31 ensure: 

7=Q and 4,^ = (f>Q = 4,^ , 

where we used (3.15), (3.28) and (3.34) for the last identity. This concludes the 
proof of ( [3^ , 

5iep ^. Strong convergence in of fn to Q . 



We first note that (3.29) and (3.26) imply 



/„ Q in L\ 



We then claim that the extra gain (3.27) -which is again a consequence of the 
monotonicity (3.4)- allows us to identify Q as the limit of /„. We indeed claim that 



( 3.27[ ) and |/„ - Ql^i ^ imply 

/ {Q ~ fn)+dxdv — )• as n — )• -|-oo. 

JR6 



(3.37) 



Let u s assume (3.37) and conclude the proof of Theorem 1.4 We first claim that 

(3.38) 



(3.37), together with fn^Q in L^, imply 

/ {fn — Q)+dxdv — )• as n — )• -|-oo. 

Indeed we observe that for all g,h£ L^(M*5) with 5 > 0, /i > 0, we have 

r+co /■ 

I vneas {g < t < K\ dt = I {h — g)j^dxdv < \g — h\]^i, (3.39) 
Jo Jms 
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< 



meas 



(/n - Q) + dxdv < ifn- fn) + dxdv + I {fn- Q) + dxdv 

{fn<t< fn}dt+\U-Q\Ll 
{fn<t<L}dt+\U-Q\Ll 
{fn - fn) + dxdv + \fn- Q\l^ 

i 

(Q - fn) + dxdv + [ {fn- Q)+ + l/n " Q|l1 



meas 



< 



< / {Q- fn) + dxdv + 2\fn-Q\L^ 



where we repeatedly used (3.39) and the fact that /„ G Eq(/„) imphes 
Mt > 0, meas |/n < i < /n | = meas |/n < i < /n | • 



and (3.38) gives 



fn ^ Q in L'^, we then conclude that (3.37) implies (3.38). Finally adding (3.37) 



l/n - QIli as n —?- +00. 



Furthermore, (3.36) and the strong convergence ^cpf^ — ^(pQ in imply: 

IbP/ralii I^PQIli as +00, 

Together with the a.e. convergence of fn, this yields the strong convergence of |t'P/n 
to Note that the uniqueness of the limit now implies the convergence of all 



the sequence /„ which completes the proof of (1.23). 



Proof of (3.31). It is a consequence of (3.27) and of the convergence: 



Jn^Q in L^- 



(3.40) 



We first claim that (3.27) remains true if one replaces (/>j^ by 

,|2 



< Tn :-- 



+ (t)Q{x) ][fn- fJ^ dxdv ^0, as n ^ +oo. (3.41) 



The fact that r„ > can be proved exactly in the same way as for (3.6), since /„ 



and fn ar e equ imeasurable. Let us now prove that Tn — s- 0. We observe from 
assumption (1.22) that Ifn'^'^li'^ = \fn\L^ — l/nlii ~^ \Q\l^, and that 

fn{s,t)^Q*{s,e), for ae s > 0, £ > 0. 

This implies that 

f*J''^{x,v) ^ Q{x,v), for ae (x,z;)eM^ 
As a consequence of the Brezis-Lieb Lemma (see [33], Theorem 1.9), we then get 

{fn^'-QW ^0. (3.42) 

We now write 
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\v\ 

T 



+ [fn- fJ'-)dxdv = 



< 



fn fn 



+ 



(t>fn{x) 



f*'Pfn _ f* 

Jn Jn 

>0, 



dxdv 



where we have used the definition (2.63) of /*''', the uniform convergence of the 



potential (j)f^, the boundedness of and /„ in the energy space, and the 
convergences (3.40) and (3.42) . Using T„ > and the convergence (3.27), we 



finally deduce that T„ — )• 0, and (3.41 ) is proved. 



Arguing as in the proof of (3.6|), we write (|3.41|) in the following equivalent form 

,|2 



+ 00 

dt 



+ 4>q{x) dxdv 



\v\ 



+ 4>q{x) dxdv 



where 



^ 0, 
(3.43) 



= {{X,V) G m.\f*J"\x,v) < t < fnix,v)}, 
S^t) = {{X,V) G R^fn{x,v) < t < f*J''{{x,v)}. 



Now from (2.74), we have 

|2 



\V\ 



+ (t>Q{x)>{f*oa^Q) ^{t,\xxv\^), 



if ix,v) E S'^it). Thus 



T > 



+ 00 



dt 



t=o 



if* o a^^) {t, \x X v\ )dxdv 



+ (boix) dxdv 



(3.44) 



As a consequence of the equimeasur ability of fn"^^ and fn, we claim that 

if - [ ] {f:oa^Q)-'{t,\xxv\^)dxdv = 0. (3.45) 
\Jssit) Js^{t)J 

Indeed, we first use the change of variables 

r = \x\, u = \v\, £ = \x X f 

to get 



(/n°a0„) ^{t,\x X v\'^)dxdv = {fri^Ho) ^{t,i)du(:{r,u)de, 



if:oa^^)-'{t,i)u,isi,mdi, 



and the same identity holds for 5*2 (t), where ui is given by (1.12), and 



Sleit) = {{r,u) G n,, f7^{r,u,t) < t < /„(r,n,£)}, 
SUt) = {ir,u) G n,, fnir,u,l) < t < f*J'' ir,u,e)}. 
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Since fn'^^ Eq(/„), we have: 



This imphes (3.45) and then (3.44) gives: 

ifn°a^Qy\t,\xxv\'^) 



T > 



dt 



t=0 



Now from (2.73), we have 



(/n°a0Q) \t,\xxv\'') 



> 



+ (t>Q{x), 



dxdv. (3.46) 



for {x,v) G S^{t). Thus, from ( |3.4lD and ( |3.46[ ), we get 



An 



ifn 



"^(t, \x X vp) 



+ 



^0, (3.47) 



as n — )• +00, for almost every {t,x,v) e M+ X R3 X r3^ We now claim that this 
implies 



Br, 



{Q* oa^^) ^{t,\x X t>|2) 









q(^)) 







%^W(^'^)^0' (3-48) 



X M'^ X M'^, where 



as n — )■ +00, for almost every {t,x,v) G 

52 (t) = {(x,!-) G m6),/„(x,z;) < t < Q{x,v)}. 
To prove we write 

= {si^\sl) u (^2" n 5^) , 5^ = (5^\52") u (52™ n 5^) , 

and get 



4 — R 



+ 4>q{x) - {Q* ° a^Q)~Ht, k X f P)^ 



+ {{fn°HQ) X t;|2) - ^ -0Q(x)j 

+ [Un ° a</,g)"^(t, k X v\'^) - {Q* o a<^g)-^(t, \x x up)] 15^(^)^52 W 

(3.49) 

We shall now examine the behavior of each of these terms when n — >• 00. We first 
observe from (3.39) and (3.42) that 

r>+00 

0, 



meas(52"(t)\52(t))dt<|/7'^-g|Li 
which implies (up to a subsequence extraction) 







1 



0, for ae {t, x, v) £ 



X X 



Using in addition the estimate 

ifn ° a0Q)(~^)(t, \x X v\'^) < le^^j^x^pl < I'Aq(0)|, 



we deduce that the first two terms of the decomposition (3.49) go to when n goes 
to infinity, for almost every (t,x,v) e M+ X R3 X We now treat the third term 
and show that 

(3.50) 
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for almost every (t, x, v). To prove (3.50), one may assume that '^s"{t)nS2{t)(-'^^ v) = 1 
for n large enough, (t, x, v) being fixed, otherwise qq = and (3.50) is proved. Let us 
also recall from standard argument that the strong convergence ( |l.22 ) together 
with the monotonicity of /* in e and the continuity of Q* in e ensure: 

a.e. i>0, VeG (e^Q,,,0), fZ{a^Q{e,£),£) ^ Q^a^^ie, £),£). 

Hence, from (1.10), we deduce that for a.e. {x,v) G M^, we have 

Ve G (e<^Q,^0), f*{a^Q{e,£),£) ^ Q*{a^Q{e, £),£), where |a; x uj^ > 0. 

(3.51) 

Let then (t, x, v) being fixed such that '^s"(t)r\S"(t)(^' v) = 1 for n large enough and 



(3.51 ) holds. From 



Q{x, v) = Q* [a^Q^i + (t>Q{x),ej ,£j >t 

and from the continuity of Q* {■,£), we deduce that 

(Q* oa^Q)-\t,£) = sup{e £]e^Q,i,0[: Q*ia^Q{e,£),£) > t}. 
Take now any e such that 

e^Q,£ < e < 0, and Q*{a^Q{e,£),i) > t, 



(3.52) 
(3.53) 



then from (3.51 ): 



for n large enough. Using the definition of the pseudoinverse given in Lemma 2.10 
we then get e < (/* o a^g)~^{t, \x x t>p) for n large enough, and hence 

e < liminf(/* o asr,)~^{t, \x x f p). 

Since this equality holds for all e satisfying ( |3.53 ), we conclude from (3.52) that 
liminf(/* o a<^ )"^(i, \x x v\^) > {Q* o a0„)~^(t, \x x v\^), 



and (3.50) is proved. 



We now turn to the decomposition (3.49) and get from (3.50) 
liminf(yln — i?„) > 0, for a.e. {t,x,v). 



Finally, observing that Bn > and using (3.47), we conclude that (3.48) holds true: 

(x) ] l|;„<t<Q| ^0, (3.54) 



(^{Q*oa^g)^-'\t,\xxv\^) 



for ae {t, x, v) in x 
Observe now that 



p6 



t<Q{x,v) implies Q{x,v) = F 



+ 



x), \x X f 1^ I > t, 



By Assumption (A) and Corollary 2.9 e — )• F{e, \x x up) is continuous and strictly 

2 



I |2 

decreasing with respect to e = '-^ — h 4>q{x) for {x,v) G {Q > 0}, and thus: 
t<Q{x,v) implies (Q* ° ^(^q)*" ""^^(^j " "''^^ 



\X X v\ 



\v\ 
~2 



x) > 0. 



We then deduce from (3.54) that 



1 



{fn<t<Q} 



0, as n — )• oo, 
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for ae (i, x, v) £ 

poo r 

/ / ^{t<Q}dxdvdt = \Q\li < +00. 
Jo Jr^ 

we may apply the dominated convergence theorem to conclude: 

as n 



X RS. Now from l{/„<t<Q} < ^{t<Q} and 



^{fn<t<Q}dxdvdt 



00. 



Injecting this into (3.39) yields (3.37) . 



This concludes the proof of Theorem 1.4 



3.4. Nonlinear stability of Q. We now turn to the proof of Theorem 1.5 which 
is a direct consequence of Theorem 1 .4 and the known regularity of strong solutions 
to the Vlasov-Poisson system. 



Proof of Theorem 1.5 Let /q G Srad H and let f{t) G Srad be the corresponding 
global strong solution to (1.1). Then from the properties of the flow of the Vlasov- 
Poisson system (1.1), there holds: 

V(/>/ e C([0, +00), l2(M^)) (3.55) 

and 

Vt>0, /(t)GEq(/o), n{f{t))=n{fo). (3.56) 

Note that /(t) G Eq(/o) means the equality of the symmetric rearrangements at 
given i: for all t > 0, f{t)* = /q. Recall also from the property of contractivity of 
the symmetric rearrangement (2.62) that: 



|/o-Q1li<I/o-QIli 



(3.57) 



Remark finally that the following inequality can be proved by interpolation: for all 
/ G Srad satisfying n{f) < n{Q) + 1, l/l^i < IQI^i + 1 and \f\L^ < M, we have 

|V(/>/-V0qU2 <Cq|/-Q|[(''. (3.58) 
Let us fix £0 > such that 

(3.59) 



5 

2 ' 



1.4 



where Cq is the constant in (3.58) and 6 is as in Theorem 

An equivalent reformulation of Theorem 1.4 is the following: for all e > 0, there 
exists r] satisfying 

< r] < min(eo, 1) 
such that the following holds true: if / G Srad is such that 

\r-Q*\L^<v, I/Il-<m, n{f)<n{Q) + v 

and 

|V</./- V0Q|i2 < ,5, 

then we have 

|/-Q|ii<min(e,eo), l/U- < M, \\v\\f-Q)\L^<e. (3.63) 

Let /o G Sradf^Cc Satisfying (1.24). From (3.56) and (3.57), we first deduce that 
the corresponding solution f{t) of (1.1) satisfies (3.61) for all t > 0. Hence, if we 
prove that 

Vt > 0, |V0/(t) - V0Q|i2 < 6, (3.64) 
then it will imply that f{t) satisfies (3.63) for all t > 0, which is nothing but ( |1.25 ). 



(3.60) 

(3.61) 
(3.62) 
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Therefore, it remains to prove (3.64). By (1.24) and (3.60), the initial data 
/(O) = /o satisfies 

I/(0)-Q|li <min(eo,l), |/(0)Uoc < M and 'H{fm<'H{Q)+rj, 



thus (3.58) and (3.59) imply that |V(/)j(0) — V<^q|^2 < 6/2. Now, assume that 
(3.64) is not true. Then there exists ti > such that |V0j(ti) — 'V(j)Q\i2 > 6 and, 
by the continuity property (3.55), there exists t2 > such that 



26/3. 



(3.65) 



\V(l)fit2)-V^Q\L2 

Hence, since the function f{t2) satisfies (3.61) and ( |3.62 ), it satisfies (3.63). There- 
fore, we have 

\f{t2)-Q\L^ <eo, |/(t2)|L-<M and -^(/(ta)) < H(Q) + t?, 



thus (3.58) and (3.59) imply that |V0/(t2) — ''^4>q\l^ ^ which contradicts 

(3.65). The proof of Theorem 1.5 is complete. □ 



4. Study of the reduced functional J' 

This section is devoted to the proof of Proposition |3.2| which requires a detailed 



study of the reduced functional J' defined by (3.8). In particular, we aim at prov- 
ing that J7 is twice differentiable at (j)Q, that (pq is a critical point and that the 
Hessian at (f)Q is definite positive. Remarkably enough, this last fact holds because 
the Hessian of is deeply connected to the structure of the linearized transport 
operator close to (pQ which is explicit in the radial setting. The coercivity of the 
corresponding Hartree-Fock exchange operator, [41], then follows from Antonov's 
celebrated coercivity property |3i i4j . 

4.1. Antonov's coercivity and the structure of the linearized transport 
operator. Let us start by describing some properties of the linearized transport 
operator generated by (pn which is deeply connected to the structure of the Hessian 



of J', see Proposition 4.3 These properties rely on standard abstract functional 



analysis results and a remarkable coercivity property due to Antonov, 1^ . 

Let denote the partial derivative with respect to e of the function F{e, t) = 
Q* (^a^g{e,i),£) defined in Assumption (A). Abusing notations, we will note when 
no confusion is possible: 

= F^x, ^,) = 1^ (^M! + \^ X . (4.1) 

Denote 



n = {{x,v) : Q(x, ?;) > 0} . (4.2) 

I |2 

At any {x,v) £ 17, we have {'-^ — h </>q(x), |x x wp) £ O, where O is defined in 
Assumption (A), hence F^{x,v) < 0. Moreover, the function {x,v) i— )• F'^{x,v) is 
continuous on 

We now consider the L? weighted Hilbert space: 

L^p,]^ = l^/ G Ljg^{Q) spherically symmetric with J dxdv < +oo^ 

and introduce an orthogonal decomposition: 

T 2,r j-2,even ^ 7- 2, odd 
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- 2,even 



|/gL^^^I with f{x,-v) = f{x,v)^, 

l\f^\^ = {/ e L^\Fi^^ with /(x, -v) = -f{x,v)^ . 
We then consider the unbounded transport operator: 



Tf = v- V,/ 



V,/, D{r) = \f ei"^^;,^, ■^/^^Ki} 



Note that C'^{^1) C D{T) is dense in L'^pi^ and hence D{T) is dense in L^p^- We 



claim the following properties of T: 



Proposition 4.1 (Properties of T). (i) Structure of the kernel; iT is a self adjoint 
operator with kernel: 

(4.3) 



(a) Coercivity of the Antonov functional; The Antonov functional 



— dxdv — I V0g 



L2 



(4.4) 



is continuous on L^piy Moreover, 

(Hi) Let g G [-^(7')]"'" H L^j^,'|"^". Then A{g,g) > and we have A{g,g) = if and 
only if g = 0. 

Proof. Step 1: Description of the kernel 

Property (i) relies on the integration of the characteristic equations associated with 
T/ = and is a standard consequence of the integrability of Newton's equation 
with central force field in radial symmetry. The proof follows similarly like for the 
proof of Jean's theorem in j8], see also |19] . 

Step 2. Proof of (ii) 

Let g G C^(il). We integrate by parts to get: 



rO > / dxdv. (4.5) 



g{x,v)4>g{x) dxdv <\g\^2,r 



{(j)g{x)f F^dxdv 



1/2 



< \g\r2,r \V(pg\L2 



where we used (C.l ) proved in the Appendix. The density of C^(il) into allows 



us to extend this estimate: 



and the continuity of (4.4) onto L'^pi^ follows. 



2,odd 



Antonov's coercivity property is now the following claim: G C^(il) n L^p,'^ 



Jn l-^el 



dxdv. 



(4.6) 



In the case where the function F depends only on e = |wp/2 + (p{x), a proof of 
this inequality can be found in |25| l23l H6l 152] . In our context F depends on 
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e = 1^1^/2 + (p{x) and i = \x x t>p, and for a sake of clarity and completeness, we 
give a proof of this inequality in Appendix |B] which is a simple extension of the 
proof in |23) . 

Let us extend this estimate to all ^ € D(7') nL^^,^*^ using standard regularization 

arguments. Let ^ G D{T) H L^p,'!^'^ and assume first that Supp(^) C 0. From the 
continuity of F^, we deduce that Fl{x,v) < 6 < for all {x,v) £ Supp(^). Let a 
mollifying sequence (n{x,v) = ^C(^)^) ^ C^(M^) with C ^ 0) then first from 
standard regularization arguments: 



Cn*C ^ Cn * (TO in L^^^i as n +oo, 



and 



T{Qn^i)^Ti, in L^Fl,\ n — > +CXD. 



Antonov's coercivity property applied to C„ G C^{Q) n L^pf^ , the continuity of 

4>' (r) 

=2/ on and the boundedness of — yield the claim. Consider now a general 

i G D{T) n L^^pf^'^. We let Xn a function such that 

' x(5)=0 fors< 
X increasing on ^] , (4.7) 



and we set 



X{s) = 1 for s > ^, 



Xn{x,v) = Xn{Q{x,v)). 



(4.8) 

Then x„ is a function with a compact support in il, satisfying Txn = 0. There- 
fore Xn^ £ L'^p'I^'^j has compact support in Q and 

HXnO = XnT^ in 
and hence the previous step and the continuity of on L'^pi\ yield (4.5). 



Step 3. Proof of ( in ) 



We first observe that the transport operator exchanges parity in v: G D{T), 



A _ r z,oa( 



_ r2,even 



/- _ r 2, even 



_ r2,odd 



This implies: 



(4.9) 



On the other hand, iT being self-adjoint, there holds -see Cor. IL17, p. 28 in |9]-: 

R(f)=N{r)^. (4.10) 



Let g G [N{T)]''~ H L^^/J'^"'. From (4.9) and (4.10), we infer the existence of a 
sequence G -0(7") n L^pn"^ such that 



g in Lf^^i 



(4.11) 



2 r 

as n — )• -|-cxD. Hence, from the continuity of the Antonov functional on L^p,^, we 
have 

^{nn,nn)^^{9,9)- (4.12) 



40 



M. LEMOU, F. MEHATS, AND P. RAPHAEL 



Moreover, by (4.5), we have 



dxdv > 0. 



(4.13) 



Thus (4.12) and (4.13) nnply £/{g,g) > 0. 



Assume now that £/{g,g) = 0. Then (4.12) and (4.13) imply that 



\F'\ 

K el 



dxdv — )• 0. 



(4.14) 



as n ^ +00. Solving the Poisson equation in radial coordinates yields: 



r (pqir) = 47r / pq{s)s ds 



Denote tq = i^f(x,v)en \x\- From the definition (4.2) of Q and the continuity of Q, 
we have a sequence rj — )■ rg, rj > tq, such that pgirj) > 0. Hence, for all r > tq, 

(/)' (r) 

we have r'^cpqi^r) > r'j(f)'Q{rj) > 0, for j large enough. Thus, the function jpzj^ is 
continuous and strictly positive on and (4.14) implies that 

^„^0 in LL(f^). 



Therefore, T^n ^ in the distribution sense T)'{Q) and, by (4.11), g = 0. This 

□ 



concludes the proof of Proposition 4.1 



A standard consequence of the explicit description of the kernel of T given by (4.3 ) 
is that we can compute the projection onto N(T) -see [19] for related statements-. 
For later use, we introduce the following homogeneous Sobolev space 

|/i G H^{R^) s.t. h is radially symmetric | . 

Lemma 4.2 (Projection onto the kernel of T). Let 

V = {{e,i) €R*_xR\: e> 



(4.15) 



where e^pQ^g is defined by (2.3). Given h € H}: , we define the projection operator: 
Vh{x, v) 



e[x,v) - (f)Q{r) ^ 



e{x,v) - 4>Q{r) ^-2- 



h{r^dr 

YJ^ '^(e(x,v)/{x,v))eV ■, (4-16) 



dr 



where ri = ri(0Q, e(x, v), £(x, f )), r2 = r2{4'Q , e{x , v) , £{x , v)) are defined by (2.6), 
(2.7), and where 

U,|2 



e(x, v) 



+ 



x), £{x, v) = \x X v\ 



Then: 



and 



{Vhm G N{T), {h - Vh)Fi G [N{T)]^ n L^^J" 



(4.17) 
(4.18) 



with given by (4.1). 

The proof is given in Appendix [C| 



41 



4.2. Differentiability of J . Our aim in this section is to prove tlie differentiability 
of J at (^Q and to compute tlie first two derivatives. We sliall in particular exhibit 



an intimate link between the Hessian of J and the projection operator (4.16). 



'J S ^rad, both nonzero. 



Proposition 4.3 (Differentiability of J). The functional J defined by (3. 
^rad satisfies the following properties. 

(i) Differentiability of J. Let (p = (pf £ ^rad 0'''^d 
Then, the functional 

\^ j{(l) + \{4>-(t>)) 

is twice differ entiahle on [0, 1] . 

(ii) Taylor expansion of J near (j)Q. Let R> and 

f e Br := {g £ £rad such that \g - Q\£ < R] . 
Then we have the following Taylor expansion near cpq: 
1 



on 



(4.19) 



'-D'J{<pQ){^f-(t>QAf 



+ eR{^f)\V<Pf-V^Q\i, (4.20) 



W/)-Wq) 

where 

ERicpf) ^0 as iVcpf - V(Pq\l2 with f £ Br, 
and where the second derivative of J in the direction h is given by 

D^Ji^Q)ih,h)= [ |V/ipdx+ / h{x)ih{x)-Vh{e,£))F^{e,e)dxdv 



(4.21) 



with Vh given by (4.16) and e 



+ 



\x X v\ 



Proof. Let us decompose J into a kinetic part and a potential part: 



^(0) = Jq.(0)=-H(Q*<^) + ^|V0 



\V(P\'^dx + Jo{(j)) (4.22) 



with 



+ 0(x)^ Q* (^a^ (^-^ + ^{x), \x X v\ 
+ 0(x) ) Q*'^{x,v)dxdv. 



\X X v\ 



dxdv 



(4.23) 

Observe that (4.23) seems to suggest that two derivatives of Jq should involve two 



derivatives of Q* and which are not available in particular from the regularity 
only of the integral (2.17) defining as- We claim that is in fact not the case and 



that suitable integration by parts and change of variables and a careful track of the 
dependence on (e, 0, t) of the various estimates on and its derivatives given by 
Lemmas [ZSl |2]4l ll]!] will yield the result. 



Step 1. Bounds for the support of Q* . 



In Corollary 2.9 we have identified the function Q*: 

Q*{s,£) = F (^a^^{s,e),i^ , V£>0, Vs > 0. (4.24) 

Recall that, by Assumption (A), for all £ > the function e i— >• F{e,i) is nonin- 
creasing. Let us define 



L = {£>0: F{e^Q,e,e)>0}, 



(4.25) 
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where e^g/ is defined in Lemma 2.1 By Lemma 2.1 (i) and by the continuity of F 
the function £ i— t- F{e^„^i 



is continuous on Ml, thus L is an open set 



If £ G \ L, then a^^{s,£) > e^^^i imphes 







for all s > 0, thus 

yi€R\\L, Q*{.,i) = 0. (4.26) 
In particular, since Q = Q*'^Q is not zero, the measure of L cannot be zero. 



Let now i ^ L and let 



where we recall the definition (2.68) of eo(^)- From Assumption (A), Lemma 2.3 



and (4.24), we infer that the function Q*{-,i) is continuous on R-(_, that its sup- 



port is [0, So(^)] and that this function is strictly decreasing and on ]0, So(^)[- 



Furthermore, from (2.21), we deduce that 



V£ G L, < so(^) < so := levr^lQl^ileor^/^ (4.27) 

Finally, let us prove that the set L is bounded. From Assumption (A), {x,v) i— t- 
Q{x,v) is compactly supported, thus there exist rQ,uo > such that Q{x,v) = 
for all (x,f) such that \x\ > tq or \v\ > uq. Hence, we have Q{x,v) = for all 
{x,v) such that \x x > rQUo and then, by definition of Q*, Q*(-,i) = for all 
£ > Iq := TqUq. Therefore, we have 

LC]0,4[. (4.28) 

Step 2. First derivative of Jq. 



We first transform the expression (4.23) of Jq. Using the change of variable (2.71) 



and the bounds (4.26), (4.27) for the support of Q* , we get 



G^-raAlO}, Jo(0)= / / al\s,l)Q''{s,£)Mds, 

Jl<^L Jo 



(4.29) 



where we recall that a^^{-,i) is defined as the inverse function of e i— )■ a^{-,i) at 
given E $rad\{0}, and £ > 0. 



Let (j) and (p as in Proposition 4.3 (i) and h = (j) — (j). Let us differentiate the 

(4.30) 



following function with respect to A G [0, 1]: 

Jo{4> + Xh)= I I a^l^,^{s,£)Q*{s,£)d£ds. 



L Jo 



Let 



g{X,s,£) = a^l,^{s,£)Q*{s,£). 
According to (2.49), we have 

dg 



{\s,£) 



'1/2 

a^+Xh^^^ ^) - ^4>A^) - ^Kr) ) h{r)dr 



ri 



-1/2 



-Q*is,£), 



dr 



where rj, i = 1,2, shortly denotes rj((/) + Xh,a^^^f^{s,£),£) defined by (2.6), (2.7) 



and ip(f)/{r) is defined by (2.2). Therefore, 



< ^(A,s,^) < \h\L^ Q*is,£) G L\R+,R+) 
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and we deduce from dominated convergence that Jq is differentiable at ^ in the 
direction h with: 



_1 \-V2 



>"2 



ri 



'1/2 



-Q*{s,i)dMs. 



Using the change of variable s e = a^^{s,£) and (2.23), we now get the following 
equivalent expression: 



DJo{(l)){h) = 47r^V2 / / Q* {a4,ie,£),e){e-2P^4r))-^/^h{r)drdede. 

(4.31) 



Step 3. Second derivative of Jq. 

Let us now compute the second derivative of j7o((/> + A/i) with respect to A. First, 
we write the first derivative in a more convenient form. Let 



V. 



4>,e 



(r, e) G 



s.t. e 



= {(r,e) G M;x]e0,^,O[ s.t. ri{4>,e,i) < r < r2{4>,e,£)} . 
An integration by parts gives 
d 



dX 



M(t} + \h) 



87r2\/2 
-87r2\/2 





L J e=e. 



<l> + \h,l 



d_ 
de 



L Jv. 



ij>+\h,e 



dQ* 
ds 



r2{(p+Xh,e,e) 



ri{(j)+\h,e,e) 



,,£(r) - Xh{r))^^^ h{r)dr 



dedi 



{a^+Xh{e,£),£) — g^(e,£) (e - ^^/(r) - \h{r)f^ h{r)drded£ 



where the boundary terms of the integration by parts vanish. Now, we perform the 
change of variable e i— )• s = a^j^\h{e,i) and get 



d_ 



with 



Jo{(l) + Xh) = -8tt'^V2 



dQ* 



L Jo 



r=0 



G{X, s, £, r)drdsd£, 



We have 

dG 1 dQ* 



(4.32) 



dX 2 ds 



is,i) 



'da'] 



<p+Xh 

dX 



s,i)-h{r) h{r) 



( -1 x-^l"^ 



(4.33) 



From (2.49): 



da 



(f)+\h 



dX 



is,i)-h{r) 



<2\h\i 



Moreover, applying (2.9) to the potential (j) + Xh gives 

'1/2 r^j2rir2 



< 



■s/{r - ri)(r2 - r) \/Z 



(4.34) 



(4.35) 
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for r G]ri,r2[. Inserting (4.34) and (4.35) in (4.33) yields 



dG 



dX 



< C\h\Loo\rh\ 



dQ* 
ds 



^/{r - ri)(r2 - r) V?' 



(4.36) 



for r G]ri, r2[, where we recall that < for £ € L and < s < So{£). 

In order to apply Lebesgue's derivation Lemma A.l one has to bound the right- 
hand side of (4.36) by suitable functions. To this aim, we need estimates for 

n = ri{(l) + Xh,a^l^f^{s, £),£), r2 = r2{<t) + Xh,a^l^^{s, £),£). 

We claim that, for all = G ^rad^ 



y£<io, ys<so{i), r2{(t>,a^\s,£),i)<C\f\i 



m^ + £o + si 



m. 



(4.37) 



where is defined by (2.20) and where C is a universal constant. Let us assume 
(4.37) and conclude the computation of the second derivative. Let 

m = mm{m^,m~) > 0, M = max(|/|ii, |/|^i). 

For A G [0, 1], we have 

m^+xh = inf (r + 1)((1 - X)\^{r)\ + A|0(r)|) > (1 - X)m^ + Xmr>m>0. 

r>0 V 

From ( |4.28[ ), ( |4.36[ ) and ( |4.37| ), we get 

dG 



dX 

ioi £ G L, s < so{£), with 

0<qx{s,£,r) 



< C\h\L'^ \rh\L^ qx{s,£,r), 



(4.38) 



dQ* 
ds 



{s,£) 



'■ri <r<r2 



y^{r - ri)(r2 - r) 



and where the constant C only depends on m, M, £q and sq. By Lemma |2.4| and 
the continuity of ri{(f) + Xh, e, £), i = 1,2 with respect to A, we have for all Aq G [0, 1] 



qx{s,£,r) ^ qxQ{s,£,r) as A — )• Aq for a.e. r,s,£. 



Moreover, 



Since Q* G 



L Jo 



qxis,£,r)drdsd£ = TT / Q* {(),£) 



d£ 

V£ 



and from (4.28), this integral is finite and its value is 



independent of A. Now we invoke the Brezis-Lieb Lemma to conclude: 

qx{s,£,r) ^ qXo{s,£,r) as A ^ Aq in L^{[0, sq] x [OJq] x R+). 



We conclude from (4.32), (4.38) and Lemma A.l that A i— )• J^o{4' + ^h) is twice 
differentiable on [0, 1]. In particular, j7o is twice differentiable at (p in the direction 
h with: 



D^Joi(l)){h,h) 



L JO 



dQ* ^ 
ds 



'da-/ 



dX 



-{s,£)-h{r) h{r) x 



X {a/{sj) - i^4,A^)) drdsd£. 



^1/2 
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By (2.49), this expression can be simplified into 



D''jo{<^){h,h) 



-47r2\/2 



L Jo 



L Jo 



soii) QQ* 

ds 

QQ* 

ds 



{s,i) {a-^\s,ll.) - V^^/r)) ^'^ {h{r)fdrdsdi 



Is, \ 



r2 



;i(5,^)-V'0,£(r) h{r)dr 



^1/2 



4> {sj) - 'ip4>/{r] 



-1/2 



-dsd£. 



dr 



(4.39) 

Proof of the claim ( 4.37 ). In order to show that ri and r2 are not allowed to go to 
infinity, we shall use (2.8). To this aim, we first need to show that e = a^^{s,i) is 
not allowed to go to zero when ^ < and s < So(^). From (2.22), we deduce that, 
for ^ < 4, s < soW, 



1 1/2 > 



mm 



2^2m^ + € 3 s 



> C min 



y/rricf, + io ' So 



(4.40) 



Finally (4.37) can be deduced from (2.8) and (4.40). 



Step 4- Identification of the first and second derivatives of J at (pq. 
Let / G £rad and h = (f)f — (pq. We claim that 

DJ{(t>q){h)=0. 



In order to prove this claim, we first remark from (4.22) that 



DJ{<i)Q){h) = DJo{(l)Q){h) + I VcPq- Vhdx. 

JR3 



(4.41) 
(4.42) 



Moreover, by (4.31), we have 
DM4>q){h) = 



47r2\/2 



F{e,£) (e - ip^Q/ir)) h{r)l^_^^ ^^^r)>odrdedl. 



where we used (2.67). Applying the change of variable e i— )• n = Y^2(e — (j)q{r)), it 
comes 



DM^q)ih) 



OO /" + 00 /" + 00 



JO JO 

Q{x, v)h{x) dxdv, 



^ ( y + ) h{r)lru>idvic 



where we used (1.10), Assumption (A), and recall that h is radially symmetric. 
Hence, from the Poisson equation, we deduce after an integration by parts that 

DJo{(kq){h) = -f V(t>q-Vhdx, 

JRS 

which together with (4.42) implies (4.41). 



Let us now identify the right second derivative of J' at (pq. We have 



D'j{(^q){h,h) = D'Jo{(^Q){h,h)+ / \Vh\'dx 



(4.43) 
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and, by ( [CTD , 
D''jo{4>Q){h,h) 



-47r2\/2 



L Jo 



dQ* 
ds 



L Jo 



ds 



r2 



-1 \-l/2 

o-cpgi^^^) -^'f'QAr)) h{r)dr 



-1/2 



-dsdl. 



dr 



Using first the change of variable s i— t- e = o^^(s,^), (2.23) and (2.67), we get 
D''M^Q){h,h) = 



L J EA 



eoW 



r2 



r2 



e - i^^PQ/i^)) iKr)fdrdedi 
{e-i;^Q/{r)) ^^'^ h{r)dr 



(e- V'</.q/(?-)) ^'"^ dr 



-dedl. 



We next apply the change of variable e i— )• n = y^2(e — 4>q{x)) and use (1.10) to 
get: 

D^Jo{^Q){h,h)= [ F^{e,i){h{x)fdxdv-[ Fi{e,i)h{x)Vh{e,£)dxdv, 



where we used the definition (4.16) and where we shortly denoted 

\v\^ 

e = — + 4>Q{x), i = \xxv\'^. 



This together with (4.43) concludes the proof of (4.21). 



Step 5. Proof of the Taylor expansion (4.20). 



We are now ready to prove the Taylor expansion (4.20). We first deduce from (4.41 ) 

h respect 

+ Xh) dX. 



and from the fact that J{(pQ + Xh) twice differentiable with respect to A that 
+ h)-JicPQ)= / (1-A) 







Hence, for h ^ 0, 



1 



JicPg + h)- J{4>q) - ^-D^J{<t>Q){h, h) 



(1 - A) {D^J{(t>Q + Xh) - D^J{<Pq)) {h, h) dX 

\Vh\l2 f\l - A) {D''Mct>Q + Xh) - D^M4>q)) ^ ^ 
Jo 



h 



We now claim the following continuity property: 

sup sup iD^Jo{(t>Q + X{4>f - 0Q) - D^Jo{(t>Q)) (h,h) 
Ae[o,i] |v/i|^2=i 



dX. 

(4.44) 

(4.45) 
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as |V0/ - V(t>Q\L2 ^ 0, / satisfying ^Jd^. 
Assume ( 4.45^ . Then: 

(1 - A) {D^McPq + Xh) - D^Joi^Q)) 



h h 



\Vh\L2'\Vh\ 



L2 



and (4.44) now yields (4.20). 



Proof of (4.45). We argue by contradiction and consider e > 0, /„ satisfying (4.19), 
hn and A„ € [0, 1] such that 



n 



and 



> e. 



(4.46) 
(4.47) 



We denote hn = Xn{4>f„ — (f'Q)- Recall from (4.39): 
D'^Joi'pQ + hn){K, K) = 



-47r2\/2 



L Jo 



ds 



{s,£) / (a^l+j^^sj) - i^^c^Ar) - hn{r)) {K{r)fdrdsM 



-1/2 



L JO 



ds 



%Q+hS^'^) -^'PqA^) -hnir)) K{r)dr 



-1/2. 



-1/2 

Ar)-hn{r)] dr 

(4.48) 



-dsdi. 



where we have denoted, for i = 1,2, 



By (4.46 ) and standard radial Sobolev embeddings, the sequence of radially symmet- 
ric functions /i„ is compact in L°°{[a,b]) for all < a < 6. By diagonal extraction, 
we deduce the pointwise convergence of (up to a subsequence) to a function h: 

Vr G hnir) h{r) as n +oo. (4.49) 

Moreover, 



-|-oo 



r^'^\hn{r)\<[ s\hAs)?ds] <\VK\l2 = 1, (4.50) 



1/2 



thus, in particular, r^/'^h belongs to 



Let us analyze the convergence of (4.48). In a first step, recalling (4.26) and 



(4.27), we fix £ e L and s e]0, so(^)] and set 



" = "0Q+/^n(*'^)' eoo = a^^(s,^) < 0. 



From (4.46), the uniform bound of /„ in E^^d and Lemma |2.4[ we have: 

Cn — Coo as n — )• +00. 



(4.51) 



For = 0, 1 or 2, we introduce the functions 



gk{n, s, £, r) = (e„ - il^ipgA^) - K{r)) {hn{r)f (r)+h„(r)<e„ (4.52) 



and 



gk{oo, s, e, r) = (coo - (Hr))'' l^^(r) 
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We claim: V£ G L, Vs g]0, so{i)], 



+00 



r+00 

gk{n,s,£,r) dr ^ / gk{(X),s,£,r) dr for = 0, 1 or 2. (4.53) 
Jo 

Indeed, from ( 4.49^ , (4.51), (4.46) and the bound of /„ which imply |/in|L°° 0, we 
first deduce that, for all s > 0, ^ > 0, the function gi:{n, s, i,r) converges pointwise 
in r S to the function (7^(00, s, r), for k = 0,1 or 2, as n — )• +00. Moreover, 
by applying (2.9) to the function cpq + hn, we get 



< gk{n,s,£,r) < 



<r<ri; 



■sj lir — r^){r2 — r) 



- I ^i^r-r'l){rl^ -r) 



< 



"<r<r" 



\J t{r — r^){r2 — r) ' 



(4.54) 



where we used (4.50). Now we observe from the uniform bound of /„ in from 
(4.46) and from (2.29) that we have 

sup IQ + A„(/„ - Q)!/,! < 00 and < inf inf (r + l)|(/)Q(r) + /i„(r)| < +00. 

„ n r>0 



Injecting the bound (4.37) on r2 into (4.54) gives: 

1 



0<gk{n,s,e,r)<K 



r\ <r<rS 



y'l{r -r'^){r^-r)' 



(4.55) 



Denote r?° 



'jVV^CJ,eoo,^) for i = 1,2 and 



^£(^r — r")(r2 — r) 



and Qooir) : = 



By Lemma 2.4, we have r" — )• r?° as n — )• +00, for i = 1,2. Therefore, the function 
Qn converges pointwise to goo- Since 

"+00 

qn{r) dr = 



+00 



3 (r) dr 



IT 



(4.56) 



we deduce from the Brezis-Lieb leinmaj that — Qqq in L-^(M+) as n — ^ +C!0. 
Finally, by applying the generalized dominated convergence theorem, we obtain 
( 4:531 ). 

Now, we remark that, with the notation ( 4.52[ ), (4.48) reads 

_ _ r rso(i) 

D^Jo{cl)Q + hn){hn,hn) = -4TT^V2 / G(n,S,^)d 

Jl Jo 



sdl 



and 



with 



D^M<i)Q){h,h) = -^^^^/2 



L Jo 



G{oo, s, £)dsd£. 



G{n,s,t) 



dQ* 
' ds 



{s,l) 



g2{n,s,£,r)dr 



gi{n, s,i,r)dr 



go{n,s,i,r)dr 
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and 



G{oo,s,£) 



dQ* 
ds 



§2(00, s,i,r)dr 



gi{oo,s,£,r)dr 



90(00, s,£,r)dr 



From (4.53), we get that, for all I £ L and s g]0, So(^)[, 

G{n, s, I) — )• G(oo, s, i) as n — )• +00. 
Moreover, by the Cauchy-Schwarz inequahty, we have 



(4.57) 



< 



gi{n,s,l,r)dr] < 



go{n,s,i,r)dr 



Therefore, (4.55), (4.56) and (4.58) yield the estimate 



g2{n,s,£,r)dr j . 

(4.58) 

(4.59) 



Remark that the function in the right-hand side of (4.59) belongs to since it is 
nonnegative and 



'"^'^ d£ 



d£ 

Q*{0,£)—<+oo, 



where we used ( |4.28[ ) and |Q*(0,^)| < \Q\l-°. Finally, we deduce from ( |4.57D , ( |4.59[ ) 
and from dominated convergence that 

D^Jo{(l>Q + K)iK,hn) ^ D^Joi(l)Q)(h,h) (4.60) 



as n — )• +00. This contradicts (4.47) and concludes the proof of (4.45). 



This concludes the proof of Proposition 4.3 



□ 



Remark 4.4. Note that a similar argument like for the proof of (4.60) gives that 
for all bounded sequence hn G H^, after extraction of a subsequence, 

D^Jo{(l)Q)(hn,hn) ^ D^M4>Q)(h,h) (4.61) 



as n — )• +00. Indeed, we never used for the proof of (4.60) the fact that is a 
Poisson field. The important consequence is then that the quadratic form D'^J'q^^'q) 
is compact on H^. 



4.3. Proof of Proposition 3.2 We are now in position to conclude the proof of 
Proposition 3.2 The coercivity property (3.9) will appear as a consequence of the 
fact that the Hessian (4.20 ) can be connected to the Antonov functional (4.4 ) via the 
projection operator (4.16), and the key step is then Antonov's coercivity property 

(4:51). 



Proof. Step 1. Strict positivity of the Hessian 



We claim that: 

V/i G \ {0}, D^J{<j)Q){h, h) > 0. 



(4.62) 
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Indeed, let h E iJ;^\{0} and consider the projection Vh given by (4.16). From (4.17) 



the functions hF^ and (Vh)F^ belong to L'^pi^ and hence g = [h — Vh)F'^ G ^\F'y 
By the orthogonality property (4.18), we have 



j {h- Vhf F^ dxdv =- j {h- Vhf F^ dxdv + 2 j h{h-Vh)F'^ dxdv 

2 j Vh- V(j)g dx, 



I 



F'\ 

^ e 



dxdv 



D^J{<t)Q){h,h) 



where we used the Poisson equation. We may thus rewrite the Hessian (4.21): 

/ |V/ipdx+ / {h-Vhf F^dxdv 

j ^ dxdv - \V(Pg\l2 + \Vh - V<l)g\l2 



Now, from (4.18) and Proposition |4.1| (in), we deduce that £/[g,g) > 0. Therefore, 
D'^J{(l>Q){h,h) is nonnegative. Moreover, if D'^J'{(j)Q){h,h) = 0, then £/{g,g) = 

(in) enables to conclude that 



\Vh — V4'g\i2 = and using again Proposition 
bg = h = 0. This ends the proof of (4.62). 



Step 2. Coercivity of the Hessian and conclusion 
In Remark 



4.4 



we have seen that the quadratic form D'^Jq{(1)q) is compact on 
H^. Hence from (4.22), the Fredholm alternative can be applied to the quadratic 
form D'^J'{(j)Q). Together with the strict positivity property (4.62), this implies the 
coercivity of this quadratic form: 

V/i G Hi D^J{cl)Q){h, h) > c\Vh\l2 , (4.63) 

for some universal constant c > 0. 

We now may conclude the proof of (3.9). Let i? > be fixed. From Proposition 
(a), there exists 6o{R) - chosen in ]0, ^|V</>Q|i2] - such that, for all / G Srad 



4.3 



satisfying 



we have 



\f-Q\£<R, \V^f-V^Q\L2<6o{R), 



< 



4' 



deduce from (4.63) and (4.20) that 



where c is the constant in (4.63) and er is defined in (4.20). Hence, for such /, we 



J{ct,Q + h)-J{^Q)>-\Vh\l2. 



The proof of Proposition 3.2 is complete. 



□ 



51 

Appendix A. A dominated convergence lemma 



Lemma A.l. Let I be an interval o/M and let g{X,r) be a real-valued function in 
C°(/,Li(M+)). Let 



G{X)= [ g{X,r)dr 
Jm.+ 



dg 



and denote by — — the weak partial derivative of g with respect to X. Assume that g 
oX 

satisfies the following assumptions: 



dg 



and for all An G /, lini -7^(X,r) 
A^Ao dX^ ' 



dg_ 
dX 



dX 



(A,r) 



(ii) for all A G /, 

Qx — ^ 'ZAq in L^{^+) as X ^ Aq. 
Then, G is on I and 

G'{X) 



< qx{r) a.e., where qx € L^{]1 



(Ao,r) for a.e. r; 
and for all Aq € /, 



dX 



(A, r)dr. 



Proof Let Aq, A G /. Since g G C^{I,L^{R+)) and |f G L^{I 



we have 



Hence, by Fubini, 

G{X) - GjXo) 
A- An 



5'(A,r) - g{Xo,r) 



g(A,r) - g(Ao,r) 
A- An 



Ao 



dX 



ifi,r)dfi. 



dr 



A - Ao Jxo 



dg_ 
dX 



(/X, r)dr I dfi 



(A.l) 

Now, we use a generalized version of the dominated convergence as stated in |33] 
(see Remark after Theorem 1.8) and deduce from Assumptions (i) and (ii) that 



hm 

fi-^Xo 



dg_ 
dX 



{n, r)dr 



dg_ 
dX 



{Xo,r)dr. 



(A.2) 



Hence, by using (A.2), we pass to the limit in (A.l) and obtain 

G(A) - G(Ao) _ 



hm 

A^o A - Ao 



dX 



{Xo,r)dr, 



which proves the differentiability of G. In fact, we observe that the same Assump- 
tions (i) and (ii) associated to the same generalized version of the dominated conver- 
gence theorem provide the continuity of G' . This ends the proof of the lemma. □ 



Appendix B. Proof of the Antonov inequality (4.6) 



Let O be defined by (|4.2 
transport operator T is 



and let ^ G C^i,^) n L^/,^''. We recall that the linear 
defined by 



Our aim is to prove the coercivity property (4.6). Let g = 7"^, we have from the 
Poisson equation 
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1 

47r 



Pg{x)dx 

\x\<.r 

r I I i'^ ' ~ ^x4>Q ■ ^vO dvdx 

™ J\x\<r JR3 



47r 

1 

47r 
1 

47r 



|a;|<r 



|a;|=r 



vS,dv I dx 



■ v^dv I da{x). 



Now we observe from the spherical symmetry of ^ that the quantity J^3 f ■ vS^di 
only depends on r = Therefore 



'^'/'o(^) = / ■ v)^dv. 



We then use the Cauchy-Schwarz inequality and Supp(^) C to estimate: 



r>;(r)^ < / (x • vy\F^\dv 



(B.l) 



(B.2) 



where we recall that is given by (4.1). Now we claim that 

{x-vf\F^\dv = r^PQ{r). (B.3) 

! 

Indeed, we first pass to spherical coordinates in v 

u = \v\, \x X v\'^ = r'^u^ sin^ 9, ^G[0,7r[, with r = |x|, (B.4) 
and get (recall that F^ < 0) 



/ {x-vf\Fi\di 



iv = — 47r 

/O Jr=0 

Now we perform the change of variable 

,2 



n 2 f+oo op / 2 

r u cos c^smt^— — — 
de \2 



+ <hoir),r'^u^ sin^ 6* ) dud6' 



„2„.2 „■ 2 , 



{u,6) ^ ( e = Y + 4>o{r)J = r'u'sin" 



and obtain 



/ (x • = -2ttV2 [ 



+00 /•+00 



=0 7e=</,Q(r)+^ 

We then integrate by parts with respect to the variable e: 



2r2 / 5e 



(B.5) 



e,i)dede. 



{x ■ vf\F^\dv = ttV2 



+ 00 f+CO 



e=0 Je=^Q{r)+, 



-1/2 



F{e,l)idedl. 



Using the same changes of variables (B.4) and (B.5), we get 
r'^pqix) = J^^F (^-^ + (I)q{x),\x xv\'^^ dv 



IT 



V~2 



OO /" + 00 



£=0 7e=0Q(r)+; 



2r2 



-1/2 



and (B.3) follows. 



53 



Now, we integrate the inequality (B.2) with respect to r and use (B.3) to get 



IV 







< 



r3 Jo 



4:Trr'^PQ{r)—-drdv 



^2 

PQ{x)---dxdv 



From the definition (4.4) of tlie Antonov functional, we then deduce 

^{Ttm > I imf - PQ{x)e) ^ dxdv. 

Jn \^e\ 
Let now ^ = (x ■ v)q{x,v) and write from the definition of T 

{TO^ = (qr{x ■ v) + {x ■ v)Tqf 

= (x • vfiTqf + {x ■ v)T{x ■ v)T{q^) + q^{T{x ■ v)f 

= {x ■ vf{T qf + T{{x- v)q^T{x ■ v)) - (x • v)q^T {T{x ■ v)) . 

We observe, from the Poisson equation A(j)Q = pq, that 



(B.6) 



T {T{x ■ v)) = -{x ■ v)A(j)Q -v-V, 



-{x-v) PQ{r) + 



Thus 



mf - PQ{r)e = {x ■ vfiTqf +T{{x- v)q^nx ■ v)) + ^^9^, 



(B.7) 



We now insert this expression into (B.6) and directly get the desired Antonov's 
inequality (4.6), provided the following claim is proved: 

T {{x ■ v)q'^T{x ■ v)) dxdv ^ as e ^ 0. (B.8) 

|x ■ 111 > 
\x\ > e 



Proof of (B.8 ); We shall in fact deal with the singularity at x • f = in the integral 
(B.8), recalling that q(x,v) = £,{x,v)/{x ■ v). We observe that the function \x\q(x,v) 
is bounded. To see this, let x 7^ and Rx be the orthogonal transformation of 
such that -Rxjfi = ei, where ei = (1,0,0)^. Then, due to the spherical symmetry, 

^{\x\ei,Rxv) /, I „ ^ .,1 / N ^('^ei,';;) i{r,\v\,ei ■ v) 
\x\q[x,v) = = uj[\x\, KxV), witha;(r, fj— — 



ei • RxV 



ei ■ V 



ei ■ V 



Now, we recall that ^ is odd in v and hence ^ is odd with respect to the last 
coordinate, thus u is bounded and so is rq. Note also that q is smooth on \x-v\ > 5, 
for all 5>0. 

Let e > 0. We have 



T ((x • v)q^T{x ■ f )) dxdv 



\X ■ V\> £^ 
\x\ > £ 

= -2e^ 



q^ (T(x • v)y 



X ■ V = £^ 

\x\ > £ 



dai{x,v) 2 



(x • v)'^q^T[x ■ v)da2{ 



X ■ V > £'^ 
\x\ = £ 



where dai{x, v) is the measure on the set {(x, v) s.t. x -v = and |x| > s} induced 
by the Lebesgue measure of M^, and da2{x) is the usual measure on the sphere 
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{x £ M^; |x| = e}. Now let > such that Supp{^) C + |fp < R"^}, 

then 



T ((x • v)(f'T{x ■ I))) dxdv 



\x\ > e 



< 2e 



< 



{T{x ■ v)y 



dai{x, v) 



+ 



X ■ V = e'^ 
\x\ > £ 



.3 \/kF+W ^ 

c 

< 2£\rq\lo.Iie,R) + -e^ 



r|r(x-?;)rdt; (icT2(x 



where we have set 

I{s,R) 



iTix-v)y 



X ■ V = e 

|a;|2 + |t.|2 < r2 



dai{x, v) 



and where we have used in the last estimate that rq is bounded and that is 
compactly supported. We claim that 



Iie,R)<CR, 



(B.9) 



where Cr is independent of e, which concludes the proof of (B.8). Indeed, we 
integrate by parts to get: 



I{e,R) 



X ■ V > e 

2 1 l„|2 / r2 



T {T{x • f )) dxdv 

X ■ V > 

|a;|2 + |t,|2 =ij2 

< / \T{T{x-v))\dxdv 

J\x\'^+\v\^<R? 



da^ix, v) 
\/\x\'^ + |f P 



+ 



|T(x • f) (rc • u — u • Vx 



daz{x,v) 



'|x|2 + |^|2=_R2 ^|a;|^_^|^| 

where da^{x,v) denotes the usual measure on the sphere |xp + |fp = R^. This 



concludes the proof of (B.9) and (4.6) 



Appendix C. Proof of Lemma 4.2 



Let us first prove that for h G ^rad-, the projection Vh given by (4.16) is well- 



defined. From Lemma 2.1 the denominator in the definition (4.16) is finite and non 



zero for {x,v) such that {e{x,v),i{x,v)) S T> and we have 



\{Vh)(x,v)\< sup \h{r)\. 



ri<r<r2 
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Let us now prove that hF^ belongs to L^p,y By a change of variable, we have 



\F^ \ dxdv 



< -C 



L JO 

50 W QQ* 



ds 



^ (a-i(,,^)-^Q(r)- — ) {h{r)fdrdsM 



L Jo 



ds 



^2 r|/i(r)p-y/rir2 
1 ^/i{r - ri)(r2 - r) 



drdsdi, 



where we used (4.26), (4.27) for the support of Q* (recall the definition (4.25) of L) 
and (2.9) for (j)Q. From (4.28), (4.37) and the radial Sobolev bound: 



+ 00 



ri/2|/i(r)| < / s\h'{s)fds] <\Vh\L2 



1/2 



we deduce that 



j \F^\ dxdv < -C\Vh\l2 



L Jo 



dQ* 
ds 



{s,i) 



r2 



zdrdsdi 



■i \J^{r - ri){r2 - r) 
< C\Vh\l2 l\*{0J)^^<C\Vh\l2. (C.l) 



Now we prove that {Vh)F^ belongs to L^p,^ The same change of variable as above 
gives 



j {Vhf \F^\ dxdv 



L Jo 



dQ* 
ds 



{s,i)- 



-1/2 



Hence, using the Cauchy-Schwarz inequality, we get 

JiVhf |Fg| dxdv < [ \Fl\ dxdv 



-1/2 



-dsdt 



dr 



which concludes the p roof o f ( |4.17 ). 
Observe now from (4.16), 



4.17) that {Vh)F^ is a L'^pn function of e{x,v) and 



l{x, v) and hence belongs to N{T) from (4.3). It remains to prove that (h — Vh)F^ 
is orthogonal to N{T)- Indeed, let 

e = e{e{x,v),e{x,v)) eN{r), 

then passing from the variables x, to the variable r,e,i, we get: 
{Vh){x, v)9{e{x, v),£{x, v))dxdv = 



+00 />0 /"r2 



■'^4>Q,l •'^1 

v)9{e{x, v),i{x, v))dxdv. 



2r' 



-1/2 



h{r)9{e,i)drdedi 
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Hence {h — Vh)F'^ and 6 are orthogonal for the L^pi]^ scalar product and (4.18) 
follows. 



This ends the proof of Lemma 4.2 
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